-- Logs begin at Wed 2022-07-20 18:35:10 UTC, end at Wed 2022-07-20 19:44:01 UTC. -- Jul 20 19:09:19.795536 dv-cinder-819149-14-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 20 19:09:19.802749 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 20 19:09:19 2022] *** Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: nodename: dv-cinder-819149-14-iscsi Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: machine: x86_64 Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: clock source: unix Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: pcre jit disabled Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: detected number of CPU cores: 4 Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: current working directory: / Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: detected binary path: /usr/bin/uwsgi-core Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: your processes number limit is 47680 Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: your memory page size is 4096 bytes Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: detected max file descriptor number: 2048 Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: lock engine: pthread robust mutexes Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: thunder lock: enabled Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 20 19:09:19.805628 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 20 19:09:19.848827 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: Python main interpreter initialized at 0x5602ade899c0 Jul 20 19:09:19.848827 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: python threads support enabled Jul 20 19:09:19.848827 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: your server socket listen backlog is limited to 100 connections Jul 20 19:09:19.848827 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: your mercy for graceful operations on workers is 90 seconds Jul 20 19:09:19.849172 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: mapped 403077 bytes (393 KB) for 2 cores Jul 20 19:09:19.849276 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: *** Operational MODE: preforking *** Jul 20 19:09:19.849552 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: *** uWSGI is running in multiple interpreter mode *** Jul 20 19:09:19.849552 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: spawned uWSGI master process (pid: 80232) Jul 20 19:09:19.849757 dv-cinder-819149-14-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 20 19:09:19.852939 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: spawned uWSGI worker 1 (pid: 80242, cores: 1) Jul 20 19:09:19.852939 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: spawned uWSGI worker 2 (pid: 80243, cores: 1) Jul 20 19:09:19.852939 dv-cinder-819149-14-iscsi devstack@keystone.service[80232]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 20 19:09:25.286485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 19:09:25.286485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: from cryptography.utils import int_from_bytes Jul 20 19:09:25.287341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 19:09:25.287341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: from cryptography.utils import int_from_bytes Jul 20 19:09:25.287938 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 19:09:25.287938 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: from cryptography.utils import int_from_bytes Jul 20 19:09:25.288427 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 19:09:25.288427 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: from cryptography.utils import int_from_bytes Jul 20 19:09:25.336712 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 19:09:25.337002 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 19:09:25.337162 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] command line args: [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 19:09:25.337401 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 19:09:25.337501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 19:09:25.337610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 19:09:25.337701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ================================================================================ {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 19:09:25.337837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] command line args: [] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 19:09:25.337918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] admin_token = **** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.338111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 19:09:25.338196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] config_dir = [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.338299 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ================================================================================ {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 19:09:25.338388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.338556 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] admin_token = **** {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.338640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] config_source = [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.338805 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] config_dir = [] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.338890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] debug = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.338995 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.339080 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.339263 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] config_source = [] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.339419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] debug = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.339508 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.339580 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.339665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.339829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.339911 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.339977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.340958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.341075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.341141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] list_limit = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.341205 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.341269 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] log_config_append = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.341327 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] list_limit = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.341327 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] log_config_append = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.341442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.341499 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.341556 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] log_dir = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.341617 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] log_dir = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.341680 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] log_file = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.341744 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] log_file = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.341808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.341866 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.341922 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.341978 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.342033 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.342104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.342298 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.342412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.342567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.342664 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.342791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.342875 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.342958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.343076 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.343157 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.343234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.343362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.343438 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.343560 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.343688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.343767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.343927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.344020 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.344164 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.344303 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.344451 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.344537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.344702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.344791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.344980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.345238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.345433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] publish_errors = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.345532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.345643 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] publish_errors = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.345732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.345889 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.346052 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.346122 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.346187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.346364 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.346448 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.346529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.346596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.346786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.346883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] standard_threads = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.347046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] standard_threads = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.347130 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.347241 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.347431 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.347529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.347598 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.347662 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.347721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] use_journal = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.347786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] use_journal = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.347990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] use_json = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.348087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] use_json = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.348149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] use_stderr = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.348213 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] use_stderr = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.348279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] use_syslog = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.348357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] use_syslog = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.348561 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.348666 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:09:25.348821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.348920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.348995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.349055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.349259 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.349363 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.349508 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.349596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.349748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.349843 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.349987 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.350069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.350727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.350816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.350888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.350957 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.351025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.351094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.351163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.351240 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.351311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.351385 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.351455 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.351523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.351630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.351789 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.351874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.352050 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.352133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.352328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.352419 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.352568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.352646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.352829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.352908 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.353085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.353162 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.353360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.353437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.353602 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.353686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.353857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.353939 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.354120 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.354199 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.354374 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.354455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.354624 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.354706 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.354857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.354942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.355086 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.355169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.355317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.355394 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.355554 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.355637 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.355802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.355878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.356050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.356135 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.356281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.356370 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.356534 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.356609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.356783 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.356870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.357034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.357168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.357276 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.357462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.357565 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.357750 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.357827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.357996 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.358072 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.358235 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.358316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.358462 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.358540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.358684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.358761 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.358917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.358993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.359150 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.359227 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.359389 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.359466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.359620 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.359709 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.359860 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.359937 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.360237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.360397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.360497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.360590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.360700 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.360802 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.360802 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.360975 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.361068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.361161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.361272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.361384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.361481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.361595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.361709 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.361859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.361954 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.362172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.362256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.362411 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.362488 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.362649 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.362774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.362884 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.362960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.363113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.363196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.363363 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.363449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.363600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.363679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.363835 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.363923 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.364064 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.364149 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.364347 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.364443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.364589 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.364673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.364822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.364900 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.365068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.365157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.365310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.365387 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.365580 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.365665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.365821 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.365910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.366071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.366147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.366312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.366395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.366548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.366632 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.366784 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.366867 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.367019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.367095 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.367262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.367340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.367499 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.367576 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.367737 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.367815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.367987 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.368062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.368202 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.368280 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.368461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.368539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.368698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.368776 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.369444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.369527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.369686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.369763 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.369939 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.370022 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.370179 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.370256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.370419 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.370500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.370648 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.370721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.370885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.370964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.371121 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.371209 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.371356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.371442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.371586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.371664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.371814 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.371896 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.372044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.372121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.372272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.372394 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.372529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.372605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.372758 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.372835 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.372999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.373079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.373234 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.373312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.373470 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.373547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.373706 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.373784 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.373961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.374039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.374210 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.374287 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.374444 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] auth.external = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.374522 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.374672 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] auth.external = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.374749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.374904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.374987 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.375185 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.375271 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.375417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] auth.password = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.375497 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.375645 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] auth.password = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.375722 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] auth.token = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.375875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] auth.token = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.375962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.376111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.376188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.376369 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.376451 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.376617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.376707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.376847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.376926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.377099 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.377222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.377337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.377427 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.377573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.377651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] credential.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.377812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] credential.caching = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.377890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.378059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.378144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.378306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.378384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.378545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.378623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.378780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.378865 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.379017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.379093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.379251 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.379328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.379519 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.379606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.379758 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.379835 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.380007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.380084 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.380264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.380411 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.380559 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.380636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.380808 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.380885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.381061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.381148 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.381310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.381387 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.381549 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] federation.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.381626 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.381790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.381879 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] federation.caching = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.382026 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.382103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.382272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.382349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.382505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.382591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.382736 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.382814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.382978 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.383055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.383216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.383301 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.383461 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.383538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.383696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.383773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.383944 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.384028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.384180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.384263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.384452 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.384680 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.385305 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity.caching = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.385498 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.385579 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.385650 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.385721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.385852 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.385960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.386091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.386199 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.386459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.386578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.386700 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.386818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.386946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.387061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.387201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.387316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.387459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.387574 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.387697 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.387819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.387934 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.388034 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.388170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.388256 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.388447 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.388527 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.388701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.388780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.388952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.389042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.389212 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.389292 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.389462 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.389540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.389736 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.389823 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.389981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.390060 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.390220 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.390298 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.390454 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.390538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.390692 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.390768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.390923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.391005 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.391160 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.391248 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.391400 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.391478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.391641 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.391724 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.391891 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.391977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.392124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.392201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.392396 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.392477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.392648 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.392737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.392884 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.392963 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.393132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.393210 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.393379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.393464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.393609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.393686 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.393847 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.393927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.394077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.394162 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.394317 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.394396 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.394555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.394640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.394795 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.394881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.395034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.395112 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.395270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.395347 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.395519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.395603 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.395766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.395843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.395995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.396073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.396224 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.396349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.396507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.396593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.396755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.396832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.396992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.397081 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.397229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.397307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.397466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.397544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.397697 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.397782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.397936 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.398014 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.398195 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.398274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.398425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.398512 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.398673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.398806 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.398921 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.398998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.399170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.399250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.399407 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.399483 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.399636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.399712 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.399871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.399949 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.400105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.400182 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.400390 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.400472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.400624 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.400704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.400870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.400947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.401116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.401193 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.401359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.401438 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.401605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.401682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.401849 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.401925 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.402079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.402158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.402324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.402402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.402562 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.402638 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.402796 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.402875 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.403031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.403107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.403264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.403341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.403510 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.403588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.403754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.403831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.403989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.404066 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.404242 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.404359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.404498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.404576 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.404742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.404819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.404976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.405055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.405210 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.405293 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.405450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.405529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.405698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.405777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.405937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.406014 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.406167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.406243 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.406406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.406485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.406646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] resource.caching = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.406723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] resource.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.406894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.406970 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.407139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.407218 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.407375 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.407452 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.407623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.407700 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.407872 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.407950 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.408106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.408183 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.408379 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.408485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.408632 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.408707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.408842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.408920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.409077 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] role.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.409156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] role.caching = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.409328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] role.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.409407 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] role.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.409560 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.409637 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.409809 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.409885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.410058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.410137 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.410304 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.410381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.410551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.410628 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.410793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.410872 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.411039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.411116 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.411283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.411368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.411528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.411605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.411758 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.411876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.412004 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.412090 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.412248 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.412378 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.412519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.412603 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.412764 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.412850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.413169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.413247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.413408 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.413486 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.413658 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.413744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.413909 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.413987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.414157 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.414234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.414407 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.414493 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.414645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.414723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.414898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.414976 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.415150 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.415235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.415388 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.415465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.415621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.415698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.416387 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.416484 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.416622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.416699 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.416875 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.416952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.417132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.417211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.417380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.417457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.417619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.417696 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.417855 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.417933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.418086 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.418163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.418321 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] token.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.418398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] token.caching = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.418557 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.418635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.418803 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.418879 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.419039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.419116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.419294 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.419373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.419541 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.419618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.419778 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.419855 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.420022 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.420101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.420267 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.420405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.420540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.420617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.420779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.420859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.421018 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.421095 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.421252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.421329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.421500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.421578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.421746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.421824 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.421977 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.422053 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.422218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.422297 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.422513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.422589 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.422781 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.connection = **** {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.422866 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.connection = **** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.423028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.423114 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.423276 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.423362 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.423515 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.423596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.423755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.423832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.423995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.424109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.424359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.424450 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.424523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.424595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.424719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.424811 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.424965 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.425043 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.425202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.425280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.425447 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.425533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.425682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.425760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.425931 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.426009 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.426168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.426253 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.426411 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.426489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.426640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.426717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.426877 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.426962 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.427111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.427189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:09:25.427280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 19:09:25.427358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 19:09:25.458539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.458733 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.458821 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.458897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.458973 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.459047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.467290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.467380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.496299 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.496435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.496552 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.496658 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.638490 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.639950 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.688382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.688563 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.688661 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.688743 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.753110 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.753335 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.753428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.753563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.753674 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.753786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.753869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.754005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.754087 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.754222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.754306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.754452 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.779339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.779485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.779576 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.779689 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.779788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.779940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.780027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.780179 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.780263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.780418 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.780500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.780635 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.780716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.780852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.792176 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.792538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.805292 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.805421 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.805533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.805647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.805751 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.805826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.805935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.806028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.806150 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.806232 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.806364 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.806446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.806600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.806691 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80242) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:09:25.823865 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.824078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 19:09:25.824277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.824442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.824542 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 19:09:25.824646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 19:09:25.824840 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.824933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.825047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 19:09:25.825157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.825317 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.825399 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.825576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.825656 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.825829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.826083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.836715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.836994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.837206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.837289 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.837484 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.837580 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.838267 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.838550 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.846274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.846719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.849589 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.849870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.850034 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.850261 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.850344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.850545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.850698 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.850831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.850983 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.851074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.851263 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.851345 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.851522 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.851602 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.851790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.851869 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.852060 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.852142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.852360 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.852467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.852653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.852919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.879449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.879951 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.881090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.881371 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.881605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.881885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.886670 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.886950 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.887180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.887262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.887458 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.887738 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.894567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.894801 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.895050 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.895331 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.899420 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.899706 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.899959 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.900042 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.900778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.900864 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.901061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.901141 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.901335 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.901416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.901603 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.901685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.901871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.901951 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.902132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.902393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.921070 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.921340 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.921661 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.921936 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.926267 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.926511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.929927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.930180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.930263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.930476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.935253 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.935528 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.939167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.939439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.942956 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.943230 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.943315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.943515 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.943596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.943804 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.943900 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.944078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.944161 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.944379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.944463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.944657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.963930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.964171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.964305 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.964568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.964665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.964838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.964920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.965110 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.972770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.973058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.974468 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.974744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.974839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.975025 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.977821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.978103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.978186 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.978932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.979028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.979216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.979308 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.979500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.979580 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.979777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.992945 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.993198 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:25.993286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.993475 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.993559 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.993756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.993847 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.994064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.994202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.994345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.994427 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.994599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.994682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:25.994876 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.013848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:26.014277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:26.017163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:26.017639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:26.020513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.020995 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.022129 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.022884 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.024041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:26.024338 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.024557 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:26.024630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.024836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.025627 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.032647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.033178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.034267 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:26.034890 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:26.037508 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.037798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.038082 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.038218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.038349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.038478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.038711 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.038993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.048232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:26.048547 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.048819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:26.048997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.049143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.049376 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.056715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:26.056990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.057268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.057395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80242) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:09:26.057544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.057670 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.057821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.057948 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.058081 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.058206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.058358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.058487 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.058615 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.058707 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.058890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.058972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.059160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.059293 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.059430 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.059512 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.059690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.059770 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.059953 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.060039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.060212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.060310 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.060502 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.060784 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80242) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:09:26.098802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5602ade899c0 pid: 80243 (default app) Jul 20 19:09:26.099281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5602ade899c0 pid: 80242 (default app) Jul 20 19:09:26.102191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b589e5a-fc17-4afa-b004-de4d62bd19f3 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:26.102486 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b589e5a-fc17-4afa-b004-de4d62bd19f3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:26.102764 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b589e5a-fc17-4afa-b004-de4d62bd19f3 None None] PATH_INFO: `/v3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:26.107633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1/1] 10.222.0.43 () {50 vars in 791 bytes} [Wed Jul 20 19:09:26 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:33.723325 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caa975-579a-4560-975e-457fa17df16b None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:33.723854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caa975-579a-4560-975e-457fa17df16b None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:33.723854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caa975-579a-4560-975e-457fa17df16b None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:33.728743 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1/2] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:33.734222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35a46e-4668-45c8-a41f-af27904b531d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:33.734614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35a46e-4668-45c8-a41f-af27904b531d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:33.734776 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35a46e-4668-45c8-a41f-af27904b531d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:33.802426 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: INFO dbcounter [None req-1a35a46e-4668-45c8-a41f-af27904b531d None None] Registered counter for database keystone Jul 20 19:09:33.824163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG oslo_db.sqlalchemy.engines [None req-1a35a46e-4668-45c8-a41f-af27904b531d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80243) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 19:09:33.830353 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writer thread running {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 19:09:34.006747 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG passlib.handlers.bcrypt [None req-1a35a46e-4668-45c8-a41f-af27904b531d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80243) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 19:09:34.007090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG passlib.handlers.bcrypt [None req-1a35a46e-4668-45c8-a41f-af27904b531d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80243) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 19:09:34.064965 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1a35a46e-4668-45c8-a41f-af27904b531d None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:34.066599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1a35a46e-4668-45c8-a41f-af27904b531d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:34.188965 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 2/3] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 20 19:09:34.362563 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9caa975-579a-4560-975e-457fa17df16b None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:34.430732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: INFO dbcounter [None req-f9caa975-579a-4560-975e-457fa17df16b None None] Registered counter for database keystone Jul 20 19:09:34.450095 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG oslo_db.sqlalchemy.engines [None req-f9caa975-579a-4560-975e-457fa17df16b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80242) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 19:09:34.456172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writer thread running {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 19:09:34.572467 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9caa975-579a-4560-975e-457fa17df16b None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:34.576736 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f897498-661f-4f27-ab02-807f62993cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:34.577398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f897498-661f-4f27-ab02-807f62993cff admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:34.577680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f897498-661f-4f27-ab02-807f62993cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:34.577957 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f897498-661f-4f27-ab02-807f62993cff admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:34.586002 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f897498-661f-4f27-ab02-807f62993cff admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:34.637734 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5f897498-661f-4f27-ab02-807f62993cff admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 19:09:34.638014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f897498-661f-4f27-ab02-807f62993cff admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:34.647474 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 2/4] 10.222.0.43 () {58 vars in 1324 bytes} [Wed Jul 20 19:09:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 19:09:34.654068 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35a46e-4668-45c8-a41f-af27904b531d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:34.677279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35a46e-4668-45c8-a41f-af27904b531d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:34.681645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfc48c-4fd8-43bb-86ae-30d1ac150acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:34.682765 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfc48c-4fd8-43bb-86ae-30d1ac150acb admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:34.683049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfc48c-4fd8-43bb-86ae-30d1ac150acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:34.683324 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfc48c-4fd8-43bb-86ae-30d1ac150acb admin admin] PATH_INFO: `/v3/projects/39f878600fda47389da52621d6e0947c` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:34.687609 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfc48c-4fd8-43bb-86ae-30d1ac150acb admin admin] RBAC: Authorizing `identity:get_project(project_id=39f878600fda47389da52621d6e0947c)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:34.737795 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: WARNING keystone.common.rbac_enforcer.enforcer [None req-94dfc48c-4fd8-43bb-86ae-30d1ac150acb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 19:09:34.738346 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfc48c-4fd8-43bb-86ae-30d1ac150acb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:34.742768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 3/5] 10.222.0.43 () {56 vars in 1207 bytes} [Wed Jul 20 19:09:34 2022] GET /identity/v3/projects/39f878600fda47389da52621d6e0947c => generated 344 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:36.764117 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdd2a7-2ada-46d2-92ae-0fe2c8eeab15 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:36.764655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdd2a7-2ada-46d2-92ae-0fe2c8eeab15 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:36.764655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdd2a7-2ada-46d2-92ae-0fe2c8eeab15 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:36.768804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 3/6] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:36.774158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41c5f7-17bd-495d-8ede-ea8e2948ca9f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:36.775154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41c5f7-17bd-495d-8ede-ea8e2948ca9f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:36.775252 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41c5f7-17bd-495d-8ede-ea8e2948ca9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:36.793874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0d41c5f7-17bd-495d-8ede-ea8e2948ca9f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:36.795402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0d41c5f7-17bd-495d-8ede-ea8e2948ca9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:36.803348 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 4/7] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 19:09:36.826023 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdd2a7-2ada-46d2-92ae-0fe2c8eeab15 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:36.838627 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdd2a7-2ada-46d2-92ae-0fe2c8eeab15 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:36.842872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e3121-2ffa-4a64-9f96-65896fa7fef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:36.843503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e3121-2ffa-4a64-9f96-65896fa7fef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:36.843792 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e3121-2ffa-4a64-9f96-65896fa7fef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:36.844072 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e3121-2ffa-4a64-9f96-65896fa7fef4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:36.851776 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e3121-2ffa-4a64-9f96-65896fa7fef4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:36.853612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e3121-2ffa-4a64-9f96-65896fa7fef4 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:36.862352 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 4/8] 10.222.0.43 () {58 vars in 1324 bytes} [Wed Jul 20 19:09:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:09:36.867656 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41c5f7-17bd-495d-8ede-ea8e2948ca9f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:36.876557 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41c5f7-17bd-495d-8ede-ea8e2948ca9f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:36.880927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5f072-4362-4e6a-a2d4-943b324b1d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:36.881775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5f072-4362-4e6a-a2d4-943b324b1d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:36.882055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5f072-4362-4e6a-a2d4-943b324b1d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:36.882328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5f072-4362-4e6a-a2d4-943b324b1d2f admin admin] PATH_INFO: `/v3/users/44b032994ab44ba0acf66376eaab2451` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:36.883790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5f072-4362-4e6a-a2d4-943b324b1d2f admin admin] RBAC: Authorizing `identity:get_user(user_id=44b032994ab44ba0acf66376eaab2451)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:36.886036 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5f072-4362-4e6a-a2d4-943b324b1d2f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:36.887590 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 5/9] 10.222.0.43 () {56 vars in 1198 bytes} [Wed Jul 20 19:09:36 2022] GET /identity/v3/users/44b032994ab44ba0acf66376eaab2451 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:38.954771 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd032161-5a93-44c3-9555-12a673efba23 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:38.955313 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd032161-5a93-44c3-9555-12a673efba23 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:38.955402 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd032161-5a93-44c3-9555-12a673efba23 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:38.956801 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 5/10] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:38.962229 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9ae1e-b81f-4c5d-8d0d-55aa51da11bc None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:38.962673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9ae1e-b81f-4c5d-8d0d-55aa51da11bc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:38.962770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9ae1e-b81f-4c5d-8d0d-55aa51da11bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:38.979595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-20b9ae1e-b81f-4c5d-8d0d-55aa51da11bc None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:38.980813 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-20b9ae1e-b81f-4c5d-8d0d-55aa51da11bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:38.991010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69935ffa-2508-4097-ab94-8f19f0d8fb4e None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:38.991297 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69935ffa-2508-4097-ab94-8f19f0d8fb4e None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:38.991573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69935ffa-2508-4097-ab94-8f19f0d8fb4e None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:38.992958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 6/11] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:38.998440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9cd8e-e555-4050-a29d-edcb45400edf None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:38.998750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9cd8e-e555-4050-a29d-edcb45400edf None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:38.999021 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9cd8e-e555-4050-a29d-edcb45400edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.037799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG passlib.handlers.bcrypt [None req-18b9cd8e-e555-4050-a29d-edcb45400edf None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80242) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 19:09:39.038219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG passlib.handlers.bcrypt [None req-18b9cd8e-e555-4050-a29d-edcb45400edf None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80242) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 19:09:39.065466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 6/12] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:39.084746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-18b9cd8e-e555-4050-a29d-edcb45400edf None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:39.086353 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-18b9cd8e-e555-4050-a29d-edcb45400edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:39.094911 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9ae1e-b81f-4c5d-8d0d-55aa51da11bc None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:39.119208 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:39.186993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 7/13] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:39.200913 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9ae1e-b81f-4c5d-8d0d-55aa51da11bc None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:39.209173 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9cd8e-e555-4050-a29d-edcb45400edf None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:39.235192 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348e087-bf7a-42cd-bdf0-70139ba4779c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:39.236008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348e087-bf7a-42cd-bdf0-70139ba4779c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.236303 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348e087-bf7a-42cd-bdf0-70139ba4779c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.236614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348e087-bf7a-42cd-bdf0-70139ba4779c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.248380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f348e087-bf7a-42cd-bdf0-70139ba4779c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:39.259644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-f348e087-bf7a-42cd-bdf0-70139ba4779c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:39.259644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:39.259644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:39.259644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:39.259644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:39.259644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:39.259644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:39.259644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:39.259644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:39.259644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:39.259644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:39.259644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:39.259644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:39.259644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:39.259644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:39.259644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:39.260723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:39.260723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:39.260723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:39.260723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:39.260723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:39.260723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:39.260723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:39.260723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:39.260723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:39.260723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:39.260723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:39.260723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:39.260723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:39.260723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:39.260723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:39.260723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:39.261528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:39.261528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:39.261528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:39.261528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:39.261528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:39.261528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:39.261528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:39.261528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:39.261528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:39.261528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:39.261528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:39.261528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:09:39.261528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 7/14] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:09:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:09:39.266766 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348e087-bf7a-42cd-bdf0-70139ba4779c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:39.311085 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9cd8e-e555-4050-a29d-edcb45400edf None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:39.313704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:39.351509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:39.351642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118ec7c-f6bf-42e3-95b6-bc1dbb19bd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:39.352540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118ec7c-f6bf-42e3-95b6-bc1dbb19bd77 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.353355 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118ec7c-f6bf-42e3-95b6-bc1dbb19bd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.353670 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118ec7c-f6bf-42e3-95b6-bc1dbb19bd77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.371085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348e087-bf7a-42cd-bdf0-70139ba4779c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:39.408220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c49e2-5095-4bbd-b48d-b345ca68c8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:39.408996 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c49e2-5095-4bbd-b48d-b345ca68c8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.409494 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c49e2-5095-4bbd-b48d-b345ca68c8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.409776 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c49e2-5095-4bbd-b48d-b345ca68c8d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.410541 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c49e2-5095-4bbd-b48d-b345ca68c8d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:39.410855 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c49e2-5095-4bbd-b48d-b345ca68c8d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:39.412259 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c49e2-5095-4bbd-b48d-b345ca68c8d6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:39.422132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c118ec7c-f6bf-42e3-95b6-bc1dbb19bd77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:39.423636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c118ec7c-f6bf-42e3-95b6-bc1dbb19bd77 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:39.424708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 8/15] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:39.430193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c49e2-5095-4bbd-b48d-b345ca68c8d6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:39.523025 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 8/16] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:09:39.529678 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118ec7c-f6bf-42e3-95b6-bc1dbb19bd77 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:39.532603 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:39.534378 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c49e2-5095-4bbd-b48d-b345ca68c8d6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:39.568239 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b467e2b-3a78-455b-af36-cfd46be8d6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:39.569119 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b467e2b-3a78-455b-af36-cfd46be8d6ee None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.569402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b467e2b-3a78-455b-af36-cfd46be8d6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.569680 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b467e2b-3a78-455b-af36-cfd46be8d6ee None admin] PATH_INFO: `/v3/users/44b032994ab44ba0acf66376eaab2451` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.570901 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b467e2b-3a78-455b-af36-cfd46be8d6ee None admin] RBAC: Authorizing `identity:get_user(user_id=44b032994ab44ba0acf66376eaab2451)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:39.572379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b467e2b-3a78-455b-af36-cfd46be8d6ee None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:39.574009 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 9/17] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:39 2022] GET /identity/v3/users/44b032994ab44ba0acf66376eaab2451 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:39.579319 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b467e2b-3a78-455b-af36-cfd46be8d6ee None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:39.594089 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:39.627648 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118ec7c-f6bf-42e3-95b6-bc1dbb19bd77 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:39.661332 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9678a1-4616-4f31-af2b-bea3a7f6cbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:39.662129 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9678a1-4616-4f31-af2b-bea3a7f6cbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.662409 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9678a1-4616-4f31-af2b-bea3a7f6cbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.662680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9678a1-4616-4f31-af2b-bea3a7f6cbe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.680992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b467e2b-3a78-455b-af36-cfd46be8d6ee None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:39.682971 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9678a1-4616-4f31-af2b-bea3a7f6cbe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:39.684463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9678a1-4616-4f31-af2b-bea3a7f6cbe8 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:39.705295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-2f9678a1-4616-4f31-af2b-bea3a7f6cbe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:09:39.705295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:39.705295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:39.705295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:39.705295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:39.705295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:39.705295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:39.705295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:39.705295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:39.705295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:39.705295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:39.705295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:39.705295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:09:39.705295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:09:39.705295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:39.706133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:39.706916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:09:39.706916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:09:39.706916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:09:39.706916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:09:39.706916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 9/18] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:09:39.712247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9678a1-4616-4f31-af2b-bea3a7f6cbe8 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:39.718962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93330c31-362e-4511-98d6-d012e0d64d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:39.719758 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93330c31-362e-4511-98d6-d012e0d64d1f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.720038 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93330c31-362e-4511-98d6-d012e0d64d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.720409 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93330c31-362e-4511-98d6-d012e0d64d1f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.722642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93330c31-362e-4511-98d6-d012e0d64d1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:39.724083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93330c31-362e-4511-98d6-d012e0d64d1f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:39.725706 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 10/19] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:39.731551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93330c31-362e-4511-98d6-d012e0d64d1f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:39.769080 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:39.815127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9678a1-4616-4f31-af2b-bea3a7f6cbe8 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:39.815351 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:39.835589 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93330c31-362e-4511-98d6-d012e0d64d1f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:39.850235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38f9b6-e089-4d44-9cee-ffcefe9b4142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:39.850913 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38f9b6-e089-4d44-9cee-ffcefe9b4142 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.851197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38f9b6-e089-4d44-9cee-ffcefe9b4142 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.851472 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38f9b6-e089-4d44-9cee-ffcefe9b4142 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.852207 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38f9b6-e089-4d44-9cee-ffcefe9b4142 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:39.852556 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38f9b6-e089-4d44-9cee-ffcefe9b4142 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:39.853945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38f9b6-e089-4d44-9cee-ffcefe9b4142 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:39.869612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292802d-91af-4522-9fc0-2116736ca9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:39.870266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292802d-91af-4522-9fc0-2116736ca9ce None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.870551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292802d-91af-4522-9fc0-2116736ca9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.870827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292802d-91af-4522-9fc0-2116736ca9ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.870912 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 10/20] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:39.871666 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292802d-91af-4522-9fc0-2116736ca9ce None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.domain.id=default, user.id=44b032994ab44ba0acf66376eaab2451) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:39.871976 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292802d-91af-4522-9fc0-2116736ca9ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:39.873452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292802d-91af-4522-9fc0-2116736ca9ce None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:39.881010 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 11/21] 10.222.0.43 () {56 vars in 1322 bytes} [Wed Jul 20 19:09:39 2022] GET /identity/v3/role_assignments?user.id=44b032994ab44ba0acf66376eaab2451&scope.domain.id=default&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:42.219451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb266d5b-5a09-4ff8-aa32-2d070ba81fca None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:42.219959 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb266d5b-5a09-4ff8-aa32-2d070ba81fca None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:42.220032 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb266d5b-5a09-4ff8-aa32-2d070ba81fca None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:42.221610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 11/22] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:42.230199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f87400-9068-4710-8b66-b3252aa0afca None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:42.230664 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f87400-9068-4710-8b66-b3252aa0afca None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:42.230744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f87400-9068-4710-8b66-b3252aa0afca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:42.257337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-45f87400-9068-4710-8b66-b3252aa0afca None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:42.258599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-45f87400-9068-4710-8b66-b3252aa0afca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:42.274941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b7c4a-e0d1-478e-8130-ddc843ef4384 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:42.275256 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b7c4a-e0d1-478e-8130-ddc843ef4384 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:42.275503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b7c4a-e0d1-478e-8130-ddc843ef4384 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:42.277121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 12/23] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:42.283591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee69602-4e53-48af-914d-a9d36ca2ee55 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:42.283881 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee69602-4e53-48af-914d-a9d36ca2ee55 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:42.284163 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee69602-4e53-48af-914d-a9d36ca2ee55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:42.306373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-7ee69602-4e53-48af-914d-a9d36ca2ee55 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:42.307591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-7ee69602-4e53-48af-914d-a9d36ca2ee55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:42.387073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 12/24] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:42.438556 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 13/25] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:42.457818 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f87400-9068-4710-8b66-b3252aa0afca None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:42.475607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee69602-4e53-48af-914d-a9d36ca2ee55 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:42.569446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:42.595821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:42.656312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f87400-9068-4710-8b66-b3252aa0afca None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:42.722495 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee69602-4e53-48af-914d-a9d36ca2ee55 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:42.744566 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf5bcb-b608-4583-befc-97af568e9ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:42.745679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf5bcb-b608-4583-befc-97af568e9ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:42.746207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf5bcb-b608-4583-befc-97af568e9ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:42.746681 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf5bcb-b608-4583-befc-97af568e9ca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:42.777194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf5bcb-b608-4583-befc-97af568e9ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:42.779048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf5bcb-b608-4583-befc-97af568e9ca8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:42.805583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba129e-b9ab-495a-af87-cb54302e4118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:42.806394 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba129e-b9ab-495a-af87-cb54302e4118 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:42.806683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba129e-b9ab-495a-af87-cb54302e4118 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:42.806959 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba129e-b9ab-495a-af87-cb54302e4118 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:42.812760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-44cf5bcb-b608-4583-befc-97af568e9ca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:09:42.812760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:42.812760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:42.812760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:42.812760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:42.812760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:42.812760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:42.812760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:42.812760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:42.812760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:42.812760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:42.812760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:42.812760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:09:42.812760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:09:42.812760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:42.813573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:42.814349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:09:42.814349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:09:42.814349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:09:42.814349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:09:42.814349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 13/26] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:42.815380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47ebbf-aecc-4b06-83d8-948be5fcc1eb None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:42.816520 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47ebbf-aecc-4b06-83d8-948be5fcc1eb None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:42.816520 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47ebbf-aecc-4b06-83d8-948be5fcc1eb None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:42.817131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 14/27] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:42.827897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47ebbf-aecc-4b06-83d8-948be5fcc1eb None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:42.832009 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba129e-b9ab-495a-af87-cb54302e4118 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:42.841812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-80ba129e-b9ab-495a-af87-cb54302e4118 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:42.841812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:42.841812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:42.841812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:42.841812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:42.841812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:42.841812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:42.841812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:42.841812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:42.841812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:42.841812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:42.841812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:42.841812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:42.841812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:42.841812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:42.841812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:42.842657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:42.842657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:42.842657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:42.842657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:42.842657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:42.842657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:42.842657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:42.842657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:42.842657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:42.842657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:42.842657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:42.842657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:42.842657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:42.842657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:42.842657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:42.842657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:42.844562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:42.844562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:42.844562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:42.844562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:42.844562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:42.844562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:42.844562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:42.844562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:42.844562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:42.844562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:42.844562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:42.844562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:09:42.844562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 14/28] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:09:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:42.844562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81eed8-e997-4aa4-b629-03e6661f83aa None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:42.847639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81eed8-e997-4aa4-b629-03e6661f83aa None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:42.847639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81eed8-e997-4aa4-b629-03e6661f83aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:42.876428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-fd81eed8-e997-4aa4-b629-03e6661f83aa None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:42.877648 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-fd81eed8-e997-4aa4-b629-03e6661f83aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:42.950773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 15/29] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:42.951820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81eed8-e997-4aa4-b629-03e6661f83aa None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:42.985585 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47ebbf-aecc-4b06-83d8-948be5fcc1eb None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:43.003098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:43.016846 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:43.027874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96a63f-5952-4867-9a51-b5aa9fde9b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:43.028961 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96a63f-5952-4867-9a51-b5aa9fde9b91 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.029459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96a63f-5952-4867-9a51-b5aa9fde9b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.029964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96a63f-5952-4867-9a51-b5aa9fde9b91 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.031130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96a63f-5952-4867-9a51-b5aa9fde9b91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:43.031677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96a63f-5952-4867-9a51-b5aa9fde9b91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:43.033503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96a63f-5952-4867-9a51-b5aa9fde9b91 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:43.052952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 15/30] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:43.054940 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb6ed8-18e4-41f5-8657-fc7b5ba18055 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.055980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb6ed8-18e4-41f5-8657-fc7b5ba18055 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.056482 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb6ed8-18e4-41f5-8657-fc7b5ba18055 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.058051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 16/31] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:43.058993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb6ed8-18e4-41f5-8657-fc7b5ba18055 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:43.096437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81eed8-e997-4aa4-b629-03e6661f83aa None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:43.136351 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b48534-180c-45f8-8497-96416cf47570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:43.137217 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b48534-180c-45f8-8497-96416cf47570 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.137598 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b48534-180c-45f8-8497-96416cf47570 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.137883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b48534-180c-45f8-8497-96416cf47570 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.138998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b48534-180c-45f8-8497-96416cf47570 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:43.139403 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b48534-180c-45f8-8497-96416cf47570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:43.141169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b48534-180c-45f8-8497-96416cf47570 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:43.158819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 16/32] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:43.159984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b48534-180c-45f8-8497-96416cf47570 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:43.177245 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb6ed8-18e4-41f5-8657-fc7b5ba18055 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:43.215083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69d8d3-5d55-43eb-bc19-d0f9d93f4ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:43.216110 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69d8d3-5d55-43eb-bc19-d0f9d93f4ae0 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.216618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69d8d3-5d55-43eb-bc19-d0f9d93f4ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.217120 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69d8d3-5d55-43eb-bc19-d0f9d93f4ae0 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.218077 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69d8d3-5d55-43eb-bc19-d0f9d93f4ae0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:43.219672 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69d8d3-5d55-43eb-bc19-d0f9d93f4ae0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:43.273433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:43.278358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b48534-180c-45f8-8497-96416cf47570 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:43.327498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbf599-e955-443c-a16b-7d34a20f6625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:43.328258 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbf599-e955-443c-a16b-7d34a20f6625 None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.328587 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbf599-e955-443c-a16b-7d34a20f6625 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.328916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbf599-e955-443c-a16b-7d34a20f6625 None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.329924 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecbf599-e955-443c-a16b-7d34a20f6625 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:43.331513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecbf599-e955-443c-a16b-7d34a20f6625 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:43.364712 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 17/33] 10.222.0.43 () {60 vars in 1131 bytes} [Wed Jul 20 19:09:43 2022] POST /identity/v3/roles => generated 225 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:43.366460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c642863-911f-4712-8cba-78d5029231c4 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.366746 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c642863-911f-4712-8cba-78d5029231c4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.367033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c642863-911f-4712-8cba-78d5029231c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.417639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1c642863-911f-4712-8cba-78d5029231c4 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:43.419398 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1c642863-911f-4712-8cba-78d5029231c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:43.484235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 17/34] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jul 20 19:09:43 2022] POST /identity/v3/projects => generated 301 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:43.490331 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbf599-e955-443c-a16b-7d34a20f6625 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:43.502756 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:43.566918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 18/35] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:43.608141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87226a76-1c24-48e5-be19-73b6eb1cff6e None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.608660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87226a76-1c24-48e5-be19-73b6eb1cff6e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.608954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87226a76-1c24-48e5-be19-73b6eb1cff6e None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.611109 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 19/36] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:43.611967 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87226a76-1c24-48e5-be19-73b6eb1cff6e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:43.636607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbf599-e955-443c-a16b-7d34a20f6625 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:43.691938 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72212e3a-7917-4730-858d-2696e678cc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:43.692891 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72212e3a-7917-4730-858d-2696e678cc58 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.693180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72212e3a-7917-4730-858d-2696e678cc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.693526 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72212e3a-7917-4730-858d-2696e678cc58 None admin] PATH_INFO: `/v3/users/44b032994ab44ba0acf66376eaab2451` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.695236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72212e3a-7917-4730-858d-2696e678cc58 None admin] RBAC: Authorizing `identity:get_user(user_id=44b032994ab44ba0acf66376eaab2451)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:43.697042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72212e3a-7917-4730-858d-2696e678cc58 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:43.704676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 18/37] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:43 2022] GET /identity/v3/users/44b032994ab44ba0acf66376eaab2451 => generated 247 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:43.706219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669d4ff-e67c-4c25-b730-5641be41410c None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.706518 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669d4ff-e67c-4c25-b730-5641be41410c None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.706797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669d4ff-e67c-4c25-b730-5641be41410c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.728071 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-3669d4ff-e67c-4c25-b730-5641be41410c None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:43.729726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-3669d4ff-e67c-4c25-b730-5641be41410c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:43.752685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:43.760631 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87226a76-1c24-48e5-be19-73b6eb1cff6e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:43.809297 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a4bde-d9bd-4284-832c-15923954b15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:43.810481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a4bde-d9bd-4284-832c-15923954b15a None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.810917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a4bde-d9bd-4284-832c-15923954b15a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.811202 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a4bde-d9bd-4284-832c-15923954b15a None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.811952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a4bde-d9bd-4284-832c-15923954b15a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:43.813485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a4bde-d9bd-4284-832c-15923954b15a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:43.818673 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 19/38] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:43.819567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669d4ff-e67c-4c25-b730-5641be41410c None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:43.840095 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 20/39] 10.222.0.43 () {60 vars in 1131 bytes} [Wed Jul 20 19:09:43 2022] POST /identity/v3/roles => generated 219 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:43.843613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a4bde-d9bd-4284-832c-15923954b15a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:43.898799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:43.962908 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669d4ff-e67c-4c25-b730-5641be41410c None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:44.007421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a4bde-d9bd-4284-832c-15923954b15a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:44.015237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4407ee1-1d42-4222-bc7e-70c9aa3f9a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:44.016086 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4407ee1-1d42-4222-bc7e-70c9aa3f9a5f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.017090 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4407ee1-1d42-4222-bc7e-70c9aa3f9a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.017376 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4407ee1-1d42-4222-bc7e-70c9aa3f9a5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.018613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4407ee1-1d42-4222-bc7e-70c9aa3f9a5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:44.020799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4407ee1-1d42-4222-bc7e-70c9aa3f9a5f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:44.022932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 20/40] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:44.029420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4407ee1-1d42-4222-bc7e-70c9aa3f9a5f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:44.051334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d69b0-5843-41d2-b93a-d0549eb31e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:44.052005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d69b0-5843-41d2-b93a-d0549eb31e5f None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.052330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d69b0-5843-41d2-b93a-d0549eb31e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.052602 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d69b0-5843-41d2-b93a-d0549eb31e5f None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.053380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d69b0-5843-41d2-b93a-d0549eb31e5f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:44.053524 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:44.055140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d69b0-5843-41d2-b93a-d0549eb31e5f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:44.077956 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 21/41] 10.222.0.43 () {60 vars in 1131 bytes} [Wed Jul 20 19:09:43 2022] POST /identity/v3/roles => generated 223 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:09:44.101262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93844f00-005b-4488-9cfe-c723748b209b None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.101262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93844f00-005b-4488-9cfe-c723748b209b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.101528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93844f00-005b-4488-9cfe-c723748b209b None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.103619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 22/42] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:44.110179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb39db-aeb4-41e8-9b30-d19b06882b6b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.110543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb39db-aeb4-41e8-9b30-d19b06882b6b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.110756 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb39db-aeb4-41e8-9b30-d19b06882b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.131247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-fbcb39db-aeb4-41e8-9b30-d19b06882b6b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:44.132607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fbcb39db-aeb4-41e8-9b30-d19b06882b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:44.167096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4407ee1-1d42-4222-bc7e-70c9aa3f9a5f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:44.211656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468f05e-9df6-4611-98fb-83fba530d388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:44.213248 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468f05e-9df6-4611-98fb-83fba530d388 None admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.213540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468f05e-9df6-4611-98fb-83fba530d388 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.213823 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468f05e-9df6-4611-98fb-83fba530d388 None admin] PATH_INFO: `/v3/domains/default/users/44b032994ab44ba0acf66376eaab2451/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.216254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468f05e-9df6-4611-98fb-83fba530d388 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=44b032994ab44ba0acf66376eaab2451, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:44.217849 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468f05e-9df6-4611-98fb-83fba530d388 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:44.235992 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 21/43] 10.222.0.43 () {58 vars in 1361 bytes} [Wed Jul 20 19:09:44 2022] PUT /identity/v3/domains/default/users/44b032994ab44ba0acf66376eaab2451/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:09:44.329271 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 23/44] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:44.353094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468f05e-9df6-4611-98fb-83fba530d388 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:44.433279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:44.458831 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468f05e-9df6-4611-98fb-83fba530d388 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:44.492592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff480a9-16d1-4ff7-b522-be1643a38236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:44.493422 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff480a9-16d1-4ff7-b522-be1643a38236 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.493705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff480a9-16d1-4ff7-b522-be1643a38236 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.493983 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff480a9-16d1-4ff7-b522-be1643a38236 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.495157 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff480a9-16d1-4ff7-b522-be1643a38236 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:44.496636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff480a9-16d1-4ff7-b522-be1643a38236 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:44.498257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 22/45] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:44.504592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb39db-aeb4-41e8-9b30-d19b06882b6b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:44.533445 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:44.619524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb39db-aeb4-41e8-9b30-d19b06882b6b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:44.659974 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0afbe-dc4a-4963-9ee9-3bb9c38029c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:44.660672 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0afbe-dc4a-4963-9ee9-3bb9c38029c1 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.660964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0afbe-dc4a-4963-9ee9-3bb9c38029c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.661248 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0afbe-dc4a-4963-9ee9-3bb9c38029c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.661978 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0afbe-dc4a-4963-9ee9-3bb9c38029c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:44.663444 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0afbe-dc4a-4963-9ee9-3bb9c38029c1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:44.691528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 24/46] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jul 20 19:09:44 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:46.254900 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec2396-26bb-4498-9b19-ffa4bc1b5db0 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:46.255445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec2396-26bb-4498-9b19-ffa4bc1b5db0 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:46.255445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec2396-26bb-4498-9b19-ffa4bc1b5db0 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:46.256987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 23/47] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:46.263445 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d66b83-6f59-4008-9151-3c93bbc5dedf None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:46.263954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d66b83-6f59-4008-9151-3c93bbc5dedf None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:46.264039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d66b83-6f59-4008-9151-3c93bbc5dedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:46.280839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-94d66b83-6f59-4008-9151-3c93bbc5dedf None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:46.282027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-94d66b83-6f59-4008-9151-3c93bbc5dedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:46.360156 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 25/48] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:46.382910 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec2396-26bb-4498-9b19-ffa4bc1b5db0 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:46.488077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec2396-26bb-4498-9b19-ffa4bc1b5db0 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:46.521705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59627d00-fae7-4d42-b851-364a0e05a3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:46.522491 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59627d00-fae7-4d42-b851-364a0e05a3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:46.522783 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59627d00-fae7-4d42-b851-364a0e05a3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:46.523068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59627d00-fae7-4d42-b851-364a0e05a3d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:46.533096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59627d00-fae7-4d42-b851-364a0e05a3d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:46.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-59627d00-fae7-4d42-b851-364a0e05a3d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:46.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:46.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:46.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:46.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:46.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:46.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:46.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:46.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:46.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:46.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:46.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:46.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:46.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:46.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:46.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:46.542194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:46.542194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:46.542194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:46.542194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:46.542194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:46.542194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:46.542194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:46.542194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:46.542194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:46.542194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:46.542194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:46.542194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:46.542194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:46.542194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:46.542194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:46.542194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:46.543040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:46.543040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:46.543040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:46.543040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:46.543040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:46.543040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:46.543040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:46.543040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:46.543040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:46.543040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:46.543040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:46.543040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:09:46.543040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 24/49] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:09:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:46.548407 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d66b83-6f59-4008-9151-3c93bbc5dedf None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:46.559156 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:46.648488 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d66b83-6f59-4008-9151-3c93bbc5dedf None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:46.682860 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ade83e-0320-47ac-99d2-adb4092b4a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:46.683533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ade83e-0320-47ac-99d2-adb4092b4a94 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:46.683823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ade83e-0320-47ac-99d2-adb4092b4a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:46.684113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ade83e-0320-47ac-99d2-adb4092b4a94 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:46.684907 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ade83e-0320-47ac-99d2-adb4092b4a94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:46.685486 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ade83e-0320-47ac-99d2-adb4092b4a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:46.686883 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ade83e-0320-47ac-99d2-adb4092b4a94 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:46.695495 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 26/50] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:46.701326 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59627d00-fae7-4d42-b851-364a0e05a3d5 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:46.705673 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:46.725758 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7332197-65ef-40f5-8e27-1b6acd79aebc None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:46.726050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7332197-65ef-40f5-8e27-1b6acd79aebc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:46.726338 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7332197-65ef-40f5-8e27-1b6acd79aebc None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:46.728245 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 27/51] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:46.733033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc9171-38d6-41a4-aa3d-c7875b7c540b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:46.733327 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc9171-38d6-41a4-aa3d-c7875b7c540b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:46.733614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc9171-38d6-41a4-aa3d-c7875b7c540b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:46.752027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-adfc9171-38d6-41a4-aa3d-c7875b7c540b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:46.753270 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-adfc9171-38d6-41a4-aa3d-c7875b7c540b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:46.804672 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59627d00-fae7-4d42-b851-364a0e05a3d5 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:46.815408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:46.820903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 28/52] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:46.844489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc9171-38d6-41a4-aa3d-c7875b7c540b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:46.846521 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd12d51-162e-4bec-b685-10701620b001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:46.847405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd12d51-162e-4bec-b685-10701620b001 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:46.847691 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd12d51-162e-4bec-b685-10701620b001 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:46.847966 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd12d51-162e-4bec-b685-10701620b001 None admin] PATH_INFO: `/v3/users/44b032994ab44ba0acf66376eaab2451` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:46.849198 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd12d51-162e-4bec-b685-10701620b001 None admin] RBAC: Authorizing `identity:get_user(user_id=44b032994ab44ba0acf66376eaab2451)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:46.850678 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd12d51-162e-4bec-b685-10701620b001 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:46.852920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 25/53] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:46 2022] GET /identity/v3/users/44b032994ab44ba0acf66376eaab2451 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:46.858414 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd12d51-162e-4bec-b685-10701620b001 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:46.927891 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:46.946700 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc9171-38d6-41a4-aa3d-c7875b7c540b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:46.958298 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd12d51-162e-4bec-b685-10701620b001 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:46.982186 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbe2d1-7bf7-4cd2-bb5d-ceb27eeb498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:46.983002 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbe2d1-7bf7-4cd2-bb5d-ceb27eeb498f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:46.983289 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbe2d1-7bf7-4cd2-bb5d-ceb27eeb498f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:46.983567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbe2d1-7bf7-4cd2-bb5d-ceb27eeb498f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:46.984879 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbe2d1-7bf7-4cd2-bb5d-ceb27eeb498f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:46.986383 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbe2d1-7bf7-4cd2-bb5d-ceb27eeb498f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:46.988096 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 29/54] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:46.993994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbe2d1-7bf7-4cd2-bb5d-ceb27eeb498f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:47.069633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:47.100664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4934695-77c3-4401-96ac-03a2463604b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:47.101466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4934695-77c3-4401-96ac-03a2463604b3 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:47.101752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4934695-77c3-4401-96ac-03a2463604b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:47.102655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4934695-77c3-4401-96ac-03a2463604b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:47.103895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4934695-77c3-4401-96ac-03a2463604b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:47.105117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbe2d1-7bf7-4cd2-bb5d-ceb27eeb498f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:47.105383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4934695-77c3-4401-96ac-03a2463604b3 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:47.107014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 26/55] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:47.112705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4934695-77c3-4401-96ac-03a2463604b3 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:47.139253 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b65b5-aec7-4404-9037-5ffd034994e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:47.139918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b65b5-aec7-4404-9037-5ffd034994e9 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:47.140201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b65b5-aec7-4404-9037-5ffd034994e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:47.140502 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b65b5-aec7-4404-9037-5ffd034994e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:47.141222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b65b5-aec7-4404-9037-5ffd034994e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:47.142760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b65b5-aec7-4404-9037-5ffd034994e9 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:47.167942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 30/56] 10.222.0.43 () {60 vars in 1140 bytes} [Wed Jul 20 19:09:46 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:09:47.227683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4934695-77c3-4401-96ac-03a2463604b3 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:47.271028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f68d2-ef92-4585-93f9-4c844a8313ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:47.271713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f68d2-ef92-4585-93f9-4c844a8313ea None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:47.271973 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f68d2-ef92-4585-93f9-4c844a8313ea None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:47.272274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f68d2-ef92-4585-93f9-4c844a8313ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:47.273133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f68d2-ef92-4585-93f9-4c844a8313ea None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.domain.id=default, user.id=44b032994ab44ba0acf66376eaab2451) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:47.273449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f68d2-ef92-4585-93f9-4c844a8313ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:47.275680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f68d2-ef92-4585-93f9-4c844a8313ea None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:47.284566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 27/57] 10.222.0.43 () {56 vars in 1322 bytes} [Wed Jul 20 19:09:47 2022] GET /identity/v3/role_assignments?user.id=44b032994ab44ba0acf66376eaab2451&scope.domain.id=default&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 530 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:49.211703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b64918-aae1-4a7e-8feb-d77f1dc7473f None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:49.212218 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b64918-aae1-4a7e-8feb-d77f1dc7473f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:49.212330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b64918-aae1-4a7e-8feb-d77f1dc7473f None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:49.213679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 31/58] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:49.218715 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740f9d7-b8f4-4749-b185-93c63767d603 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:49.219119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740f9d7-b8f4-4749-b185-93c63767d603 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:49.219287 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740f9d7-b8f4-4749-b185-93c63767d603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:49.225338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:49.236119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-3740f9d7-b8f4-4749-b185-93c63767d603 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:49.237376 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-3740f9d7-b8f4-4749-b185-93c63767d603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:49.303543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 28/59] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:49.328420 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b64918-aae1-4a7e-8feb-d77f1dc7473f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:49.429849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b64918-aae1-4a7e-8feb-d77f1dc7473f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:49.450484 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:49.465772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437116c6-ec49-4453-b540-0dd277aa4389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:49.466503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437116c6-ec49-4453-b540-0dd277aa4389 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:49.466795 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437116c6-ec49-4453-b540-0dd277aa4389 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:49.467077 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437116c6-ec49-4453-b540-0dd277aa4389 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:49.468305 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437116c6-ec49-4453-b540-0dd277aa4389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:49.469783 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437116c6-ec49-4453-b540-0dd277aa4389 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:49.471414 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 32/60] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:49.477419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740f9d7-b8f4-4749-b185-93c63767d603 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:49.574505 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740f9d7-b8f4-4749-b185-93c63767d603 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:49.607443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653941c-a36f-4a91-9e56-905ed2b0ac82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:49.608122 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653941c-a36f-4a91-9e56-905ed2b0ac82 None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:49.608440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653941c-a36f-4a91-9e56-905ed2b0ac82 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:49.608721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653941c-a36f-4a91-9e56-905ed2b0ac82 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:49.609449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653941c-a36f-4a91-9e56-905ed2b0ac82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:49.610884 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653941c-a36f-4a91-9e56-905ed2b0ac82 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:49.621470 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:49.639591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 29/61] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:09:49 2022] POST /identity/v3/users => generated 275 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:52.819396 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fddd6-505d-4a47-bcbd-f5313f77c58c None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:52.820318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fddd6-505d-4a47-bcbd-f5313f77c58c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:52.820787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fddd6-505d-4a47-bcbd-f5313f77c58c None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:52.822361 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 33/62] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:52.833489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97846e14-1985-4c58-892d-814335e8fdd3 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:52.834330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97846e14-1985-4c58-892d-814335e8fdd3 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:52.834782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97846e14-1985-4c58-892d-814335e8fdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:52.861357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-97846e14-1985-4c58-892d-814335e8fdd3 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:52.862837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-97846e14-1985-4c58-892d-814335e8fdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:52.953700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 30/63] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:52.989055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fddd6-505d-4a47-bcbd-f5313f77c58c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:53.039041 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e360a-d37d-49d4-ab80-340f3b2bc196 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.040097 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e360a-d37d-49d4-ab80-340f3b2bc196 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.040578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e360a-d37d-49d4-ab80-340f3b2bc196 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.042125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 31/64] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:53.048102 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334bb43-95ef-4986-b4df-37d48bea4c90 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.048639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334bb43-95ef-4986-b4df-37d48bea4c90 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.049131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334bb43-95ef-4986-b4df-37d48bea4c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.075068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-5334bb43-95ef-4986-b4df-37d48bea4c90 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:53.078018 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5334bb43-95ef-4986-b4df-37d48bea4c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:53.148351 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fddd6-505d-4a47-bcbd-f5313f77c58c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:53.218111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c07a2-dd5d-4d53-9945-21ee45e80c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:53.219167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c07a2-dd5d-4d53-9945-21ee45e80c26 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.219598 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c07a2-dd5d-4d53-9945-21ee45e80c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.219700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 32/65] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:53.220161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c07a2-dd5d-4d53-9945-21ee45e80c26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.221067 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6a539-13ba-477d-b210-d0f80cf72295 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.221525 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6a539-13ba-477d-b210-d0f80cf72295 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.221947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6a539-13ba-477d-b210-d0f80cf72295 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.224513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 33/66] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:53.236922 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77f6db-6b7c-41ab-960e-ae32e6518591 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.237461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77f6db-6b7c-41ab-960e-ae32e6518591 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.237884 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77f6db-6b7c-41ab-960e-ae32e6518591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.239623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c07a2-dd5d-4d53-9945-21ee45e80c26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:53.248381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-338c07a2-dd5d-4d53-9945-21ee45e80c26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:53.248381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:53.248381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:53.248381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:53.248381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:53.248381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:53.248381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:53.248381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:53.248381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:53.248381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:53.248381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:53.248381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:53.248381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:53.248381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:53.248381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:53.248381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:53.249432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:53.249432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:53.249432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:53.249432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:53.249432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:53.249432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:53.249432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:53.249432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:53.249432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:53.249432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:53.249432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:53.249432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:53.249432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:53.249432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:53.249432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:53.249432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:53.250440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:53.250440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:53.250440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:53.250440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:53.250440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:53.250440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:53.250440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:53.250440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:53.250440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:53.250440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:53.250440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:53.250440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:09:53.251929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 34/67] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:53.256927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c07a2-dd5d-4d53-9945-21ee45e80c26 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:53.265977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-0e77f6db-6b7c-41ab-960e-ae32e6518591 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:53.267401 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0e77f6db-6b7c-41ab-960e-ae32e6518591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:53.317105 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:53.319194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:53.353742 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 34/68] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:53.354878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77f6db-6b7c-41ab-960e-ae32e6518591 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:53.378665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c07a2-dd5d-4d53-9945-21ee45e80c26 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:53.414607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c1b5a-208b-4f26-ba18-c81f73e55038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:53.415794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c1b5a-208b-4f26-ba18-c81f73e55038 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.415794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c1b5a-208b-4f26-ba18-c81f73e55038 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.417899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c1b5a-208b-4f26-ba18-c81f73e55038 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.434657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c1b5a-208b-4f26-ba18-c81f73e55038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:53.442752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-c95c1b5a-208b-4f26-ba18-c81f73e55038 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:53.442752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:53.442752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:53.442752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:53.442752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:53.442752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:53.442752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:53.442752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:53.442752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:53.442752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:53.442752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:53.442752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:53.442752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:53.442752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:53.442752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:53.442752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:53.443605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:53.443605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:53.443605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:53.443605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:53.443605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:53.443605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:53.443605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:53.443605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:53.443605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:53.443605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:53.443605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:53.443605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:53.443605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:53.443605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:53.443605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:53.443605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:53.449893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:53.449893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:53.449893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:53.449893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:53.449893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:53.449893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:53.449893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:53.449893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:53.449893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:53.449893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:53.449893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:53.449893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:09:53.449893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 35/69] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:09:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:53.449893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266e682-9894-471b-a83f-f77715f00999 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.450584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266e682-9894-471b-a83f-f77715f00999 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.450584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266e682-9894-471b-a83f-f77715f00999 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.450584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 36/70] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:53.450584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266e682-9894-471b-a83f-f77715f00999 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:53.478368 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77f6db-6b7c-41ab-960e-ae32e6518591 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:53.513921 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b266e-faca-419c-a078-98546abd7b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:53.514598 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b266e-faca-419c-a078-98546abd7b61 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.514887 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b266e-faca-419c-a078-98546abd7b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.515168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b266e-faca-419c-a078-98546abd7b61 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.515937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b266e-faca-419c-a078-98546abd7b61 None admin] RBAC: Adding query filter params (name=member) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:53.516252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b266e-faca-419c-a078-98546abd7b61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:53.517694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b266e-faca-419c-a078-98546abd7b61 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:53.526970 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 35/71] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:53.528318 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6b9f3-8633-4e8d-878a-beb447d988bb None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.528621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6b9f3-8633-4e8d-878a-beb447d988bb None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.528908 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6b9f3-8633-4e8d-878a-beb447d988bb None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.530178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 36/72] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:53.530884 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6b9f3-8633-4e8d-878a-beb447d988bb None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:53.559271 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266e682-9894-471b-a83f-f77715f00999 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:53.568602 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:53.579459 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:53.595306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527f529-0fd0-48a6-895c-f165b1ec8f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:53.596111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527f529-0fd0-48a6-895c-f165b1ec8f06 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.596423 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527f529-0fd0-48a6-895c-f165b1ec8f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.596710 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527f529-0fd0-48a6-895c-f165b1ec8f06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.606856 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527f529-0fd0-48a6-895c-f165b1ec8f06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:53.615257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-0527f529-0fd0-48a6-895c-f165b1ec8f06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:53.615257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:53.615257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:53.615257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:53.615257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:53.615257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:53.615257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:53.615257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:53.615257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:53.615257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:53.615257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:53.615257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:53.615257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:53.615257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:53.615257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:53.615257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:53.616135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:53.616135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:53.616135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:53.616135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:53.616135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:53.616135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:53.616135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:53.616135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:53.616135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:53.616135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:53.616135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:53.616135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:53.616135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:53.616135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:53.616135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:53.616135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:53.616924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:53.616924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:53.616924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:53.616924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:53.616924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:53.616924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:53.616924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:53.616924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:53.616924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:53.616924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:53.616924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:53.616924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:09:53.616924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 37/73] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:09:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:53.618490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beff640-fc03-4123-a071-e53582cca4d6 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.618583 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beff640-fc03-4123-a071-e53582cca4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.618858 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beff640-fc03-4123-a071-e53582cca4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.636217 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-9beff640-fc03-4123-a071-e53582cca4d6 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:53.637466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9beff640-fc03-4123-a071-e53582cca4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:53.639925 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6b9f3-8633-4e8d-878a-beb447d988bb None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:53.674544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bdba0-ae32-4cca-a497-f0ab6a7e8482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:53.675225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bdba0-ae32-4cca-a497-f0ab6a7e8482 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.675518 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bdba0-ae32-4cca-a497-f0ab6a7e8482 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.675796 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bdba0-ae32-4cca-a497-f0ab6a7e8482 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.676602 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bdba0-ae32-4cca-a497-f0ab6a7e8482 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:53.676934 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bdba0-ae32-4cca-a497-f0ab6a7e8482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:53.678326 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bdba0-ae32-4cca-a497-f0ab6a7e8482 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:53.688000 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 37/74] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:53.688879 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bdba0-ae32-4cca-a497-f0ab6a7e8482 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:53.707113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 38/75] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:53.709648 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5deb6-9630-478e-9c86-27c40dd7cfef None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.709940 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5deb6-9630-478e-9c86-27c40dd7cfef None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.710344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5deb6-9630-478e-9c86-27c40dd7cfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.727895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4ee5deb6-9630-478e-9c86-27c40dd7cfef None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:53.729130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4ee5deb6-9630-478e-9c86-27c40dd7cfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:53.800369 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 39/76] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:53.801230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5deb6-9630-478e-9c86-27c40dd7cfef None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:53.815224 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bdba0-ae32-4cca-a497-f0ab6a7e8482 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:53.818389 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:53.834449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:53.857851 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a29bbc-296b-486a-8514-0778e468d6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:53.858643 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a29bbc-296b-486a-8514-0778e468d6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.858929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a29bbc-296b-486a-8514-0778e468d6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.859207 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a29bbc-296b-486a-8514-0778e468d6c3 None admin] PATH_INFO: `/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.875023 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a29bbc-296b-486a-8514-0778e468d6c3 None admin] RBAC: Authorizing `identity:get_user(user_id=ff6ae2c3a54f48faabac6aeb4ab6d81f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:53.876523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a29bbc-296b-486a-8514-0778e468d6c3 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:53.878246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 38/77] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:53 2022] GET /identity/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:53.879020 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a29bbc-296b-486a-8514-0778e468d6c3 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:53.914867 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5deb6-9630-478e-9c86-27c40dd7cfef None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:53.954723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0d1aa-00be-43b1-9f65-0d7f65d89f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:53.955398 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0d1aa-00be-43b1-9f65-0d7f65d89f12 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.955685 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0d1aa-00be-43b1-9f65-0d7f65d89f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.955964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0d1aa-00be-43b1-9f65-0d7f65d89f12 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.956763 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0d1aa-00be-43b1-9f65-0d7f65d89f12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:53.957071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0d1aa-00be-43b1-9f65-0d7f65d89f12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:53.958480 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0d1aa-00be-43b1-9f65-0d7f65d89f12 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:53.966686 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 40/78] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:09:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:53.972044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0d1aa-00be-43b1-9f65-0d7f65d89f12 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:53.981495 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a29bbc-296b-486a-8514-0778e468d6c3 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:54.028450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ef835-7358-4044-b4a9-c347dd94b505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:54.029333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ef835-7358-4044-b4a9-c347dd94b505 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.029617 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ef835-7358-4044-b4a9-c347dd94b505 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.029892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ef835-7358-4044-b4a9-c347dd94b505 None admin] PATH_INFO: `/v3/users/44b032994ab44ba0acf66376eaab2451` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.031123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ef835-7358-4044-b4a9-c347dd94b505 None admin] RBAC: Authorizing `identity:get_user(user_id=44b032994ab44ba0acf66376eaab2451)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:54.032633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ef835-7358-4044-b4a9-c347dd94b505 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:54.034284 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 39/79] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:53 2022] GET /identity/v3/users/44b032994ab44ba0acf66376eaab2451 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:54.034284 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ef835-7358-4044-b4a9-c347dd94b505 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:54.073070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:54.073891 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:54.084678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0d1aa-00be-43b1-9f65-0d7f65d89f12 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:54.119012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d9953-3577-471c-8a22-7fc64cb7505e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:54.119732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d9953-3577-471c-8a22-7fc64cb7505e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.120018 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d9953-3577-471c-8a22-7fc64cb7505e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.120348 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d9953-3577-471c-8a22-7fc64cb7505e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.130609 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d9953-3577-471c-8a22-7fc64cb7505e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:54.138358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-695d9953-3577-471c-8a22-7fc64cb7505e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:54.138358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:54.138358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:54.138358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:54.138358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:54.138358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:54.138358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:54.138358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:54.138358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:54.138358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:54.138358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:54.138358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:54.138358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:54.138358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:54.138358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:54.138358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:54.139174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:54.139174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:54.139174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:54.139174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:54.139174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:54.139174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:54.139174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:54.139174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:54.139174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:54.139174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:54.139174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:54.139174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:54.139174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:54.139174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:54.139174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:54.139174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:54.140278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:54.140278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:54.140278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:54.140278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:54.140278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:54.140278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:54.140278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:54.140278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:54.140278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:54.140278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:54.140278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:54.140278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:09:54.140278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 41/80] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:54.141075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d9953-3577-471c-8a22-7fc64cb7505e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:54.148928 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ef835-7358-4044-b4a9-c347dd94b505 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:54.185895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271fefe-9bd1-4887-bc13-0a00b3a2ff7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:54.186703 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271fefe-9bd1-4887-bc13-0a00b3a2ff7d None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.186995 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271fefe-9bd1-4887-bc13-0a00b3a2ff7d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.187278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271fefe-9bd1-4887-bc13-0a00b3a2ff7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.188527 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271fefe-9bd1-4887-bc13-0a00b3a2ff7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:54.189967 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271fefe-9bd1-4887-bc13-0a00b3a2ff7d None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:54.191604 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 40/81] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:54.192403 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271fefe-9bd1-4887-bc13-0a00b3a2ff7d None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:54.245002 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d9953-3577-471c-8a22-7fc64cb7505e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:54.285599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de266f-cbcb-43d0-81d4-6918cb3f494b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:54.286367 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de266f-cbcb-43d0-81d4-6918cb3f494b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.286725 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de266f-cbcb-43d0-81d4-6918cb3f494b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.287073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de266f-cbcb-43d0-81d4-6918cb3f494b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.296890 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:54.302154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271fefe-9bd1-4887-bc13-0a00b3a2ff7d None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:54.308219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:54.347121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4936f9-dceb-4efd-a8c1-ba4f9b76daba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:54.348070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4936f9-dceb-4efd-a8c1-ba4f9b76daba None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.348330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4936f9-dceb-4efd-a8c1-ba4f9b76daba None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.348616 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4936f9-dceb-4efd-a8c1-ba4f9b76daba None admin] PATH_INFO: `/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.349883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4936f9-dceb-4efd-a8c1-ba4f9b76daba None admin] RBAC: Authorizing `identity:get_user(user_id=ff6ae2c3a54f48faabac6aeb4ab6d81f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:54.351334 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4936f9-dceb-4efd-a8c1-ba4f9b76daba None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:54.353665 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 41/82] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:54 2022] GET /identity/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:54.354441 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4936f9-dceb-4efd-a8c1-ba4f9b76daba None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:54.365645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de266f-cbcb-43d0-81d4-6918cb3f494b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:54.367147 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de266f-cbcb-43d0-81d4-6918cb3f494b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:54.455011 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4936f9-dceb-4efd-a8c1-ba4f9b76daba None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:54.468557 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 42/83] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:54.469418 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de266f-cbcb-43d0-81d4-6918cb3f494b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:54.491085 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418fb1a3-8d4c-4120-8ef2-0bdbd7a8a8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:54.491743 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418fb1a3-8d4c-4120-8ef2-0bdbd7a8a8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.492035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418fb1a3-8d4c-4120-8ef2-0bdbd7a8a8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.492346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418fb1a3-8d4c-4120-8ef2-0bdbd7a8a8a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.520908 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:54.528316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:54.566463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418fb1a3-8d4c-4120-8ef2-0bdbd7a8a8a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:54.568439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418fb1a3-8d4c-4120-8ef2-0bdbd7a8a8a9 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:54.569603 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de266f-cbcb-43d0-81d4-6918cb3f494b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:54.609438 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0be88f-a79a-47d3-a265-ef96f62b2d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:54.610114 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0be88f-a79a-47d3-a265-ef96f62b2d81 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.610410 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0be88f-a79a-47d3-a265-ef96f62b2d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.610691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0be88f-a79a-47d3-a265-ef96f62b2d81 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.611446 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0be88f-a79a-47d3-a265-ef96f62b2d81 None admin] RBAC: Adding query filter params (name=member) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:54.611760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0be88f-a79a-47d3-a265-ef96f62b2d81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:54.613230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0be88f-a79a-47d3-a265-ef96f62b2d81 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:54.622182 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 43/84] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:54.622656 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0be88f-a79a-47d3-a265-ef96f62b2d81 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:54.683196 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 42/85] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:54.684065 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418fb1a3-8d4c-4120-8ef2-0bdbd7a8a8a9 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:54.744390 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0be88f-a79a-47d3-a265-ef96f62b2d81 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:54.751756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:54.758491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:54.780485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50698d49-f3cd-4efb-81c1-a7d777ff3ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:54.781164 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50698d49-f3cd-4efb-81c1-a7d777ff3ce5 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.781451 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50698d49-f3cd-4efb-81c1-a7d777ff3ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.781938 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50698d49-f3cd-4efb-81c1-a7d777ff3ce5 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.782817 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50698d49-f3cd-4efb-81c1-a7d777ff3ce5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:54.784280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50698d49-f3cd-4efb-81c1-a7d777ff3ce5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:54.787969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418fb1a3-8d4c-4120-8ef2-0bdbd7a8a8a9 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:54.810455 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 44/86] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:09:54 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:54.811303 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50698d49-f3cd-4efb-81c1-a7d777ff3ce5 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:54.840259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef1f98-c3d1-4b56-92e5-8c089d927b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:54.841346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef1f98-c3d1-4b56-92e5-8c089d927b6f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.841682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef1f98-c3d1-4b56-92e5-8c089d927b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.841972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef1f98-c3d1-4b56-92e5-8c089d927b6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.926682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef1f98-c3d1-4b56-92e5-8c089d927b6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:54.928387 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef1f98-c3d1-4b56-92e5-8c089d927b6f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:54.947866 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50698d49-f3cd-4efb-81c1-a7d777ff3ce5 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:54.991996 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3987e-9de8-4957-97ce-1391af9c36a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:54.991996 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3987e-9de8-4957-97ce-1391af9c36a8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.991996 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3987e-9de8-4957-97ce-1391af9c36a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.991996 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3987e-9de8-4957-97ce-1391af9c36a8 None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.993657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3987e-9de8-4957-97ce-1391af9c36a8 None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:54.995213 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3987e-9de8-4957-97ce-1391af9c36a8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:54.996956 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 45/87] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:54 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:54.997856 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3987e-9de8-4957-97ce-1391af9c36a8 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:55.011377 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:55.038713 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:55.058044 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 43/88] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:55.060557 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef1f98-c3d1-4b56-92e5-8c089d927b6f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:55.137584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3987e-9de8-4957-97ce-1391af9c36a8 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:55.186011 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3d4c2-02dd-413a-b234-59e33a088756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:55.186208 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef1f98-c3d1-4b56-92e5-8c089d927b6f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:55.187351 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3d4c2-02dd-413a-b234-59e33a088756 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:55.187963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3d4c2-02dd-413a-b234-59e33a088756 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:55.189296 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3d4c2-02dd-413a-b234-59e33a088756 None admin] PATH_INFO: `/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:55.190872 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3d4c2-02dd-413a-b234-59e33a088756 None admin] RBAC: Authorizing `identity:get_user(user_id=ff6ae2c3a54f48faabac6aeb4ab6d81f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:55.192586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3d4c2-02dd-413a-b234-59e33a088756 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:55.195001 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 46/89] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:54 2022] GET /identity/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:55.196987 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3d4c2-02dd-413a-b234-59e33a088756 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:55.223228 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a26948-ce72-4b64-8e0e-5883d920a5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:55.224102 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a26948-ce72-4b64-8e0e-5883d920a5ee None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:55.224432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a26948-ce72-4b64-8e0e-5883d920a5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:55.224719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a26948-ce72-4b64-8e0e-5883d920a5ee None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:55.225994 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a26948-ce72-4b64-8e0e-5883d920a5ee None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:55.227461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a26948-ce72-4b64-8e0e-5883d920a5ee None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:55.232860 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 44/90] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:55 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:55.232860 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a26948-ce72-4b64-8e0e-5883d920a5ee None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:55.277595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:55.331343 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3d4c2-02dd-413a-b234-59e33a088756 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:55.334923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:55.362943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a26948-ce72-4b64-8e0e-5883d920a5ee None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:55.382298 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db4f7c-789a-4717-81d5-8305e1e4c911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:55.383928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db4f7c-789a-4717-81d5-8305e1e4c911 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:55.384225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db4f7c-789a-4717-81d5-8305e1e4c911 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:55.384559 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db4f7c-789a-4717-81d5-8305e1e4c911 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:55.385427 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db4f7c-789a-4717-81d5-8305e1e4c911 None admin] RBAC: Adding query filter params (role.id=18f0dc41b78c4b52b6912225928ea388, scope.project.id=711e8b07480d478a80cbefc6fe30467f, user.id=ff6ae2c3a54f48faabac6aeb4ab6d81f) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:55.385749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db4f7c-789a-4717-81d5-8305e1e4c911 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:55.387227 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db4f7c-789a-4717-81d5-8305e1e4c911 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:55.404356 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 47/91] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:55 2022] GET /identity/v3/role_assignments?user.id=ff6ae2c3a54f48faabac6aeb4ab6d81f&scope.project.id=711e8b07480d478a80cbefc6fe30467f&role.id=18f0dc41b78c4b52b6912225928ea388 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:55.404356 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db4f7c-789a-4717-81d5-8305e1e4c911 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:55.419058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd24b1-f89e-41cf-aa7b-24a0f65cebb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:55.420315 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd24b1-f89e-41cf-aa7b-24a0f65cebb8 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:55.420804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd24b1-f89e-41cf-aa7b-24a0f65cebb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:55.421277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd24b1-f89e-41cf-aa7b-24a0f65cebb8 None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:55.425286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd24b1-f89e-41cf-aa7b-24a0f65cebb8 None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:55.426904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd24b1-f89e-41cf-aa7b-24a0f65cebb8 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:55.428938 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 45/92] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:55 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:55.429639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd24b1-f89e-41cf-aa7b-24a0f65cebb8 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:55.548903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db4f7c-789a-4717-81d5-8305e1e4c911 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:55.552156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd24b1-f89e-41cf-aa7b-24a0f65cebb8 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:55.554210 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:55.593376 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479df8e-f1c7-4da7-bbb3-76a6e4e3b5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:55.594254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479df8e-f1c7-4da7-bbb3-76a6e4e3b5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:55.594635 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479df8e-f1c7-4da7-bbb3-76a6e4e3b5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:55.594982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479df8e-f1c7-4da7-bbb3-76a6e4e3b5a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:55.595847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479df8e-f1c7-4da7-bbb3-76a6e4e3b5a3 None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=711e8b07480d478a80cbefc6fe30467f, user.id=44b032994ab44ba0acf66376eaab2451) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:55.596207 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479df8e-f1c7-4da7-bbb3-76a6e4e3b5a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:55.597960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479df8e-f1c7-4da7-bbb3-76a6e4e3b5a3 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:55.605808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf2c20-3886-416e-a396-d563b626f4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:55.607330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf2c20-3886-416e-a396-d563b626f4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:55.607617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf2c20-3886-416e-a396-d563b626f4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:55.607896 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf2c20-3886-416e-a396-d563b626f4b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:55.610480 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 46/93] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:55 2022] GET /identity/v3/role_assignments?user.id=44b032994ab44ba0acf66376eaab2451&scope.project.id=711e8b07480d478a80cbefc6fe30467f&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:55.611424 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479df8e-f1c7-4da7-bbb3-76a6e4e3b5a3 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:55.706294 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:55.774503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf2c20-3886-416e-a396-d563b626f4b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:55.776078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf2c20-3886-416e-a396-d563b626f4b8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:55.863011 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479df8e-f1c7-4da7-bbb3-76a6e4e3b5a3 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:55.901684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 48/94] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:55.907250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf2c20-3886-416e-a396-d563b626f4b8 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:55.932369 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dbeb6-bf23-464e-809e-d16bf436564e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:55.933051 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dbeb6-bf23-464e-809e-d16bf436564e None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:55.933344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dbeb6-bf23-464e-809e-d16bf436564e None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:55.933626 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dbeb6-bf23-464e-809e-d16bf436564e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:55.934493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7dbeb6-bf23-464e-809e-d16bf436564e None admin] RBAC: Adding query filter params (role.id=abe200f23fc84ba397dd8c67874f0f2a, scope.project.id=711e8b07480d478a80cbefc6fe30467f, user.id=ff6ae2c3a54f48faabac6aeb4ab6d81f) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:55.935069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7dbeb6-bf23-464e-809e-d16bf436564e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:55.936604 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7dbeb6-bf23-464e-809e-d16bf436564e None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:55.956384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 47/95] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:55 2022] GET /identity/v3/role_assignments?user.id=ff6ae2c3a54f48faabac6aeb4ab6d81f&scope.project.id=711e8b07480d478a80cbefc6fe30467f&role.id=abe200f23fc84ba397dd8c67874f0f2a => generated 259 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:56.043399 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:56.104237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf2c20-3886-416e-a396-d563b626f4b8 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:56.156215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a5496-e6c7-4642-9e7e-690721dcee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:56.157047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a5496-e6c7-4642-9e7e-690721dcee1a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:56.157359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a5496-e6c7-4642-9e7e-690721dcee1a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:56.157694 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a5496-e6c7-4642-9e7e-690721dcee1a None admin] PATH_INFO: `/v3/projects/249c9c15f2044b2891c272c99f0701d6` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:56.165291 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a5496-e6c7-4642-9e7e-690721dcee1a None admin] RBAC: Authorizing `identity:get_project(project_id=249c9c15f2044b2891c272c99f0701d6)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:56.166823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a5496-e6c7-4642-9e7e-690721dcee1a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:56.168698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 49/96] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:55 2022] GET /identity/v3/projects/249c9c15f2044b2891c272c99f0701d6 => generated 312 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:56.174761 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dbeb6-bf23-464e-809e-d16bf436564e None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:56.210961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:56.303686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dbeb6-bf23-464e-809e-d16bf436564e None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:56.347188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d0027-61a4-4ef6-9e79-3a7e02764a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:56.347733 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d0027-61a4-4ef6-9e79-3a7e02764a78 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:56.348242 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d0027-61a4-4ef6-9e79-3a7e02764a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:56.348359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d0027-61a4-4ef6-9e79-3a7e02764a78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:56.349377 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d0027-61a4-4ef6-9e79-3a7e02764a78 None admin] RBAC: Adding query filter params (role.id=18f0dc41b78c4b52b6912225928ea388, scope.project.id=249c9c15f2044b2891c272c99f0701d6, user.id=ff6ae2c3a54f48faabac6aeb4ab6d81f) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:56.349556 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d0027-61a4-4ef6-9e79-3a7e02764a78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:56.351761 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d0027-61a4-4ef6-9e79-3a7e02764a78 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:56.363258 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 48/97] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:56 2022] GET /identity/v3/role_assignments?user.id=ff6ae2c3a54f48faabac6aeb4ab6d81f&scope.project.id=249c9c15f2044b2891c272c99f0701d6&role.id=18f0dc41b78c4b52b6912225928ea388 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:57.765494 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66f61a-d2b7-445d-b568-3d2d509285a5 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:57.765984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66f61a-d2b7-445d-b568-3d2d509285a5 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:57.765984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66f61a-d2b7-445d-b568-3d2d509285a5 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:57.767520 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 50/98] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:57.773742 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc89a8c-5bca-4f33-b34f-d1c00373421a None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:57.774218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc89a8c-5bca-4f33-b34f-d1c00373421a None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:57.774295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc89a8c-5bca-4f33-b34f-d1c00373421a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:57.797393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-7fc89a8c-5bca-4f33-b34f-d1c00373421a None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:57.798614 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-7fc89a8c-5bca-4f33-b34f-d1c00373421a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:57.896317 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 49/99] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:57.932005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66f61a-d2b7-445d-b568-3d2d509285a5 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:58.071609 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66f61a-d2b7-445d-b568-3d2d509285a5 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:58.116275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705bb089-a23a-4acc-9a1a-04690915e5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:58.117137 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bb089-a23a-4acc-9a1a-04690915e5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.117424 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bb089-a23a-4acc-9a1a-04690915e5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.117894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bb089-a23a-4acc-9a1a-04690915e5b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.125041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705bb089-a23a-4acc-9a1a-04690915e5b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:58.126516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705bb089-a23a-4acc-9a1a-04690915e5b9 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:58.128266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 51/100] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:58.134188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc89a8c-5bca-4f33-b34f-d1c00373421a None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:58.176838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:58.272030 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795d066-b606-48e0-9046-7a054e5e23ad None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.272030 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795d066-b606-48e0-9046-7a054e5e23ad None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.272030 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795d066-b606-48e0-9046-7a054e5e23ad None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.274132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 52/101] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:58.279513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b2005-c0ba-4b28-98b4-64d36bba2fbe None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.279803 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b2005-c0ba-4b28-98b4-64d36bba2fbe None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.280085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b2005-c0ba-4b28-98b4-64d36bba2fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.286692 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc89a8c-5bca-4f33-b34f-d1c00373421a None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:58.296323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:58.318741 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-905b2005-c0ba-4b28-98b4-64d36bba2fbe None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:58.320026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-905b2005-c0ba-4b28-98b4-64d36bba2fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:58.335656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230483b2-0b8b-45f1-a5e7-82d3e0238f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:58.337247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230483b2-0b8b-45f1-a5e7-82d3e0238f6b None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.337753 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230483b2-0b8b-45f1-a5e7-82d3e0238f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.338242 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230483b2-0b8b-45f1-a5e7-82d3e0238f6b None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.339189 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230483b2-0b8b-45f1-a5e7-82d3e0238f6b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:58.340750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230483b2-0b8b-45f1-a5e7-82d3e0238f6b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:58.384325 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 50/102] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jul 20 19:09:58 2022] POST /identity/v3/projects => generated 302 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:58.384325 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7de8e9-f4b0-474d-b894-f58c2ef6d40d None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.384757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7de8e9-f4b0-474d-b894-f58c2ef6d40d None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.385007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7de8e9-f4b0-474d-b894-f58c2ef6d40d None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.388945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 51/103] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:58.388945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f3867-fac2-43d0-b74f-52e0092254fd None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.388945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f3867-fac2-43d0-b74f-52e0092254fd None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.388945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f3867-fac2-43d0-b74f-52e0092254fd None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.393895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 52/104] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:58.395560 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fcfd0-4785-43fb-83b0-6d312a97d9b6 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.396373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fcfd0-4785-43fb-83b0-6d312a97d9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.396675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fcfd0-4785-43fb-83b0-6d312a97d9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.418064 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-654fcfd0-4785-43fb-83b0-6d312a97d9b6 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:58.419569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-654fcfd0-4785-43fb-83b0-6d312a97d9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:58.430760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 53/105] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:58.432738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55313082-22fa-48b2-9d87-f08baf6e2a25 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.433331 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55313082-22fa-48b2-9d87-f08baf6e2a25 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.433776 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55313082-22fa-48b2-9d87-f08baf6e2a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.465766 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-55313082-22fa-48b2-9d87-f08baf6e2a25 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:58.470132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-55313082-22fa-48b2-9d87-f08baf6e2a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:58.513098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 53/106] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:58.513795 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654fcfd0-4785-43fb-83b0-6d312a97d9b6 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:58.530495 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:58.670166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 54/107] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:58.671461 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55313082-22fa-48b2-9d87-f08baf6e2a25 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:58.681506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654fcfd0-4785-43fb-83b0-6d312a97d9b6 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:58.732872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682812c-605d-4a20-8433-806a6a024d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:58.733969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682812c-605d-4a20-8433-806a6a024d0e None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.734446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682812c-605d-4a20-8433-806a6a024d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.734901 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682812c-605d-4a20-8433-806a6a024d0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.746609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682812c-605d-4a20-8433-806a6a024d0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:58.752882 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:58.755281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-c682812c-605d-4a20-8433-806a6a024d0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:58.755281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:58.755281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:58.755281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:58.755281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:58.755281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:58.755281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:58.755281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:58.755281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:58.755281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:58.755281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:58.755281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:58.755281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:58.755281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:58.755281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:58.755281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:58.756606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:58.756606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:58.756606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:58.756606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:58.756606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:58.756606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:58.756606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:58.756606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:58.756606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:58.756606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:58.756606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:58.756606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:58.756606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:58.756606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:58.756606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:58.756606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:58.757581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:58.757581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:58.757581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:58.757581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:58.757581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:58.757581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:58.757581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:58.757581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:58.757581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:58.757581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:58.757581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:58.757581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:09:58.759197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 54/108] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:09:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:58.760067 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682812c-605d-4a20-8433-806a6a024d0e None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:58.799721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55313082-22fa-48b2-9d87-f08baf6e2a25 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:58.832806 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:58.838966 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bb254-b825-48b7-a75b-16c42330c7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:58.840027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bb254-b825-48b7-a75b-16c42330c7bc None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.840525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bb254-b825-48b7-a75b-16c42330c7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.840985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bb254-b825-48b7-a75b-16c42330c7bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.852707 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937bb254-b825-48b7-a75b-16c42330c7bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:58.860874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-937bb254-b825-48b7-a75b-16c42330c7bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:58.860874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:58.860874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:58.860874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:58.860874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:58.860874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:58.860874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:58.860874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:58.860874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:58.860874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:58.860874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:58.860874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:58.860874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:58.860874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:58.860874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:58.860874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:58.861792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:58.861792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:58.861792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:58.861792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:58.861792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:58.861792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:58.861792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:58.861792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:58.861792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:58.861792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:58.861792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:58.861792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:58.861792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:58.861792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:58.861792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:58.861792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:58.862537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:58.862537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:58.862537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:58.862537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:58.862537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:58.862537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:58.862537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:58.862537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:58.862537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:58.862537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:58.862537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:58.862537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:09:58.864651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 55/109] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:58.865611 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bb254-b825-48b7-a75b-16c42330c7bc None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:58.874418 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682812c-605d-4a20-8433-806a6a024d0e None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:58.910768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22efc2-b3eb-488f-a7bb-4ac022feb665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:58.911580 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22efc2-b3eb-488f-a7bb-4ac022feb665 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.911866 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22efc2-b3eb-488f-a7bb-4ac022feb665 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.912143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22efc2-b3eb-488f-a7bb-4ac022feb665 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.924539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22efc2-b3eb-488f-a7bb-4ac022feb665 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:58.933075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-cf22efc2-b3eb-488f-a7bb-4ac022feb665 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:58.933075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:58.933075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:58.933075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:58.933075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:58.933075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:58.933075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:58.933075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:58.933075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:58.933075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:58.933075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:58.933075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:58.933075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:58.933075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:58.933075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:58.933075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:58.934171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:58.934171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:58.934171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:58.934171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:58.934171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:58.934171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:58.934171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:58.934171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:58.934171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:58.934171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:58.934171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:58.934171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:58.934171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:58.934171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:58.934171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:58.934171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:58.935274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:58.935274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:58.935274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:58.935274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:58.935274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:58.935274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:58.935274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:58.935274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:58.935274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:58.935274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:58.935274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:58.935274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:09:58.935274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 55/110] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:58.935894 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22efc2-b3eb-488f-a7bb-4ac022feb665 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:58.989920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bb254-b825-48b7-a75b-16c42330c7bc None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.016888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:59.028786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e72cf-a453-4e6a-a85a-9a2033c9fcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.029536 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e72cf-a453-4e6a-a85a-9a2033c9fcbe None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.029835 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e72cf-a453-4e6a-a85a-9a2033c9fcbe None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.030125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e72cf-a453-4e6a-a85a-9a2033c9fcbe None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.030904 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e72cf-a453-4e6a-a85a-9a2033c9fcbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:59.031227 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e72cf-a453-4e6a-a85a-9a2033c9fcbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:59.032693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e72cf-a453-4e6a-a85a-9a2033c9fcbe None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:59.044212 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 56/111] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:59.045337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e72cf-a453-4e6a-a85a-9a2033c9fcbe None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:59.057449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22efc2-b3eb-488f-a7bb-4ac022feb665 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.085509 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:59.095340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d655e-7a1f-4170-ad77-50172e379a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.096238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d655e-7a1f-4170-ad77-50172e379a64 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.096890 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d655e-7a1f-4170-ad77-50172e379a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.097312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d655e-7a1f-4170-ad77-50172e379a64 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.098247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d655e-7a1f-4170-ad77-50172e379a64 None admin] RBAC: Adding query filter params (name=member) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:59.098699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d655e-7a1f-4170-ad77-50172e379a64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:59.100224 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d655e-7a1f-4170-ad77-50172e379a64 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:59.110342 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 56/112] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:59.112127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e1d9a-b89d-4850-ac36-a32e7f2f9d97 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.112626 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e1d9a-b89d-4850-ac36-a32e7f2f9d97 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.113318 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e1d9a-b89d-4850-ac36-a32e7f2f9d97 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.114821 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 57/113] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:59.116278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e1d9a-b89d-4850-ac36-a32e7f2f9d97 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:59.156586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e72cf-a453-4e6a-a85a-9a2033c9fcbe None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.193717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576a6f8-36cd-47de-9bea-9bfbedab00d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.194329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576a6f8-36cd-47de-9bea-9bfbedab00d0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.194620 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576a6f8-36cd-47de-9bea-9bfbedab00d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.194908 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576a6f8-36cd-47de-9bea-9bfbedab00d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.196038 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576a6f8-36cd-47de-9bea-9bfbedab00d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:59.196154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576a6f8-36cd-47de-9bea-9bfbedab00d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:59.197545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576a6f8-36cd-47de-9bea-9bfbedab00d0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:59.206622 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 57/114] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:59.212078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576a6f8-36cd-47de-9bea-9bfbedab00d0 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:59.229073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e1d9a-b89d-4850-ac36-a32e7f2f9d97 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.265772 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1edb0-9714-4097-bb53-343cfd9d6dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.267118 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1edb0-9714-4097-bb53-343cfd9d6dec None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.267626 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1edb0-9714-4097-bb53-343cfd9d6dec None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.268093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1edb0-9714-4097-bb53-343cfd9d6dec None admin] PATH_INFO: `/v3/users/44b032994ab44ba0acf66376eaab2451` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.269617 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1edb0-9714-4097-bb53-343cfd9d6dec None admin] RBAC: Authorizing `identity:get_user(user_id=44b032994ab44ba0acf66376eaab2451)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:59.271361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1edb0-9714-4097-bb53-343cfd9d6dec None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:59.273368 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 58/115] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:59 2022] GET /identity/v3/users/44b032994ab44ba0acf66376eaab2451 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:59.273900 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:59.275199 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd91b0c-e832-48ea-9905-f36b820ba6d1 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.275695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd91b0c-e832-48ea-9905-f36b820ba6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.276240 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd91b0c-e832-48ea-9905-f36b820ba6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.294471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-cfd91b0c-e832-48ea-9905-f36b820ba6d1 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:59.296009 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-cfd91b0c-e832-48ea-9905-f36b820ba6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:59.323059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576a6f8-36cd-47de-9bea-9bfbedab00d0 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.356885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:59.359489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7707854-5235-411a-952b-c3ad57a20137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.360491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7707854-5235-411a-952b-c3ad57a20137 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.360784 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7707854-5235-411a-952b-c3ad57a20137 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.361123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7707854-5235-411a-952b-c3ad57a20137 None admin] PATH_INFO: `/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.363130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7707854-5235-411a-952b-c3ad57a20137 None admin] RBAC: Authorizing `identity:get_user(user_id=ff6ae2c3a54f48faabac6aeb4ab6d81f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:59.364629 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7707854-5235-411a-952b-c3ad57a20137 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:59.366564 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 58/116] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:59 2022] GET /identity/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:59.367920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7707854-5235-411a-952b-c3ad57a20137 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:59.378503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 59/117] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:59.388576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd91b0c-e832-48ea-9905-f36b820ba6d1 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:59.490882 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7707854-5235-411a-952b-c3ad57a20137 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.506242 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd91b0c-e832-48ea-9905-f36b820ba6d1 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.511736 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:59.536594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b61f4-8e75-455e-9e17-1e4f01384194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.537800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b61f4-8e75-455e-9e17-1e4f01384194 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.538095 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b61f4-8e75-455e-9e17-1e4f01384194 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.538379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b61f4-8e75-455e-9e17-1e4f01384194 None admin] PATH_INFO: `/v3/users/717605caf5e34451b2fac90d3a2c249a` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.546089 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d646fcc-f4e6-40e7-a21d-4787eea354f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.547248 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d646fcc-f4e6-40e7-a21d-4787eea354f6 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.547678 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d646fcc-f4e6-40e7-a21d-4787eea354f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.548115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d646fcc-f4e6-40e7-a21d-4787eea354f6 None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.550197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d646fcc-f4e6-40e7-a21d-4787eea354f6 None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:59.552061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d646fcc-f4e6-40e7-a21d-4787eea354f6 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:59.559009 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b61f4-8e75-455e-9e17-1e4f01384194 None admin] RBAC: Authorizing `identity:get_user(user_id=717605caf5e34451b2fac90d3a2c249a)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:59.559009 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b61f4-8e75-455e-9e17-1e4f01384194 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:59.559009 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 59/118] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:59 2022] GET /identity/v3/users/717605caf5e34451b2fac90d3a2c249a => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:59.559307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b61f4-8e75-455e-9e17-1e4f01384194 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:59.559986 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 60/119] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:59 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:59.561019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d646fcc-f4e6-40e7-a21d-4787eea354f6 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:59.623331 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:59.672659 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b61f4-8e75-455e-9e17-1e4f01384194 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.678270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d646fcc-f4e6-40e7-a21d-4787eea354f6 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.708445 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a35871-5bdc-420d-aac6-0871289324fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.709305 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a35871-5bdc-420d-aac6-0871289324fc None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.709598 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a35871-5bdc-420d-aac6-0871289324fc None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.709874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a35871-5bdc-420d-aac6-0871289324fc None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.711615 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a35871-5bdc-420d-aac6-0871289324fc None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:59.713077 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a35871-5bdc-420d-aac6-0871289324fc None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:59.713492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244cef9-7160-4e98-8ffb-931778a0787c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.714308 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244cef9-7160-4e98-8ffb-931778a0787c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.714598 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244cef9-7160-4e98-8ffb-931778a0787c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.719627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 60/120] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:59 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:59.719757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244cef9-7160-4e98-8ffb-931778a0787c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.719834 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a35871-5bdc-420d-aac6-0871289324fc None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:59.730632 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244cef9-7160-4e98-8ffb-931778a0787c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:59.738896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-1244cef9-7160-4e98-8ffb-931778a0787c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:59.738896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:59.738896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:59.738896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:59.738896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:59.738896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:59.738896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:59.738896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:59.738896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:59.738896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:59.738896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:59.738896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:59.738896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:59.738896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:59.738896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:59.738896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:59.739732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:59.739732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:59.739732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:59.739732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:59.739732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:59.739732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:59.739732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:59.739732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:59.739732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:59.739732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:59.739732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:59.739732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:59.739732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:59.739732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:59.739732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:59.739732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:59.740529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:59.740529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:59.740529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:59.740529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:59.740529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:59.740529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:59.740529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:59.740529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:59.740529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:59.740529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:59.740529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:59.740529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:09:59.740529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 61/121] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:09:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:59.745779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244cef9-7160-4e98-8ffb-931778a0787c None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:59.763597 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:59.825925 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a35871-5bdc-420d-aac6-0871289324fc None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.857718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244cef9-7160-4e98-8ffb-931778a0787c None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.864523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64125f6a-af8c-4f3b-b93d-10e41e78b35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.865550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64125f6a-af8c-4f3b-b93d-10e41e78b35b None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.865842 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64125f6a-af8c-4f3b-b93d-10e41e78b35b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.866120 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64125f6a-af8c-4f3b-b93d-10e41e78b35b None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f/users/44b032994ab44ba0acf66376eaab2451/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.868357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64125f6a-af8c-4f3b-b93d-10e41e78b35b None admin] RBAC: Authorizing `identity:create_grant(project_id=711e8b07480d478a80cbefc6fe30467f, user_id=44b032994ab44ba0acf66376eaab2451, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:59.869969 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64125f6a-af8c-4f3b-b93d-10e41e78b35b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:59.873286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:59.896330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 61/122] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:09:59 2022] PUT /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f/users/44b032994ab44ba0acf66376eaab2451/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:09:59.896330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b654bd7-2a1f-4e73-9cc9-709ddeb8cfc4 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.896330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b654bd7-2a1f-4e73-9cc9-709ddeb8cfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.896330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b654bd7-2a1f-4e73-9cc9-709ddeb8cfc4 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.896330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 62/123] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:59.896330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b654bd7-2a1f-4e73-9cc9-709ddeb8cfc4 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:59.908240 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea292db4-6480-4a00-9746-46979b95151c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.908952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea292db4-6480-4a00-9746-46979b95151c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.909311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea292db4-6480-4a00-9746-46979b95151c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.909716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea292db4-6480-4a00-9746-46979b95151c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.013444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea292db4-6480-4a00-9746-46979b95151c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:00.015256 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea292db4-6480-4a00-9746-46979b95151c None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:00.030050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b654bd7-2a1f-4e73-9cc9-709ddeb8cfc4 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:00.031920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:00.077417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bf134-fef7-46a1-ad51-f3b70f147378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:00.078323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bf134-fef7-46a1-ad51-f3b70f147378 None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.078614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bf134-fef7-46a1-ad51-f3b70f147378 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.078894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bf134-fef7-46a1-ad51-f3b70f147378 None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f/users/ff6ae2c3a54f48faabac6aeb4ab6d81f/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.080961 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187bf134-fef7-46a1-ad51-f3b70f147378 None admin] RBAC: Authorizing `identity:create_grant(project_id=711e8b07480d478a80cbefc6fe30467f, user_id=ff6ae2c3a54f48faabac6aeb4ab6d81f, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:00.082429 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187bf134-fef7-46a1-ad51-f3b70f147378 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:00.103873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 63/124] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:09:59 2022] PUT /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f/users/ff6ae2c3a54f48faabac6aeb4ab6d81f/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:00.104562 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bf134-fef7-46a1-ad51-f3b70f147378 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:00.148225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 62/125] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:00.150212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b16f9-7967-401d-82c5-4f7d15ab1349 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.150440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b16f9-7967-401d-82c5-4f7d15ab1349 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.150721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b16f9-7967-401d-82c5-4f7d15ab1349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.169396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-f15b16f9-7967-401d-82c5-4f7d15ab1349 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:00.170748 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f15b16f9-7967-401d-82c5-4f7d15ab1349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:00.181683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:00.254222 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 63/126] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:00.255252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b16f9-7967-401d-82c5-4f7d15ab1349 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:00.261257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bf134-fef7-46a1-ad51-f3b70f147378 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:00.352692 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edafd62-ba6b-4a63-8d92-f8403f886394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:00.353617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edafd62-ba6b-4a63-8d92-f8403f886394 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.354102 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edafd62-ba6b-4a63-8d92-f8403f886394 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.354687 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edafd62-ba6b-4a63-8d92-f8403f886394 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.355651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edafd62-ba6b-4a63-8d92-f8403f886394 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:00.356171 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edafd62-ba6b-4a63-8d92-f8403f886394 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:00.358037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edafd62-ba6b-4a63-8d92-f8403f886394 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:00.368863 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 64/127] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:00.370168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edafd62-ba6b-4a63-8d92-f8403f886394 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:00.398587 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:00.410235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b16f9-7967-401d-82c5-4f7d15ab1349 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:00.448584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d6580-97c7-4ef2-bd33-c9e4f7230dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:00.449698 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6580-97c7-4ef2-bd33-c9e4f7230dab None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.450117 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6580-97c7-4ef2-bd33-c9e4f7230dab None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.450544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6580-97c7-4ef2-bd33-c9e4f7230dab None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.451924 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d6580-97c7-4ef2-bd33-c9e4f7230dab None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:00.453672 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d6580-97c7-4ef2-bd33-c9e4f7230dab None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:00.455639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 64/128] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:00.456616 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d6580-97c7-4ef2-bd33-c9e4f7230dab None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:00.473200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:00.490899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edafd62-ba6b-4a63-8d92-f8403f886394 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:00.529360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62167b-6854-41db-afd0-1431e58af4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:00.530932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62167b-6854-41db-afd0-1431e58af4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.531324 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62167b-6854-41db-afd0-1431e58af4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.531798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62167b-6854-41db-afd0-1431e58af4a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.546474 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd62167b-6854-41db-afd0-1431e58af4a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:00.557647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-bd62167b-6854-41db-afd0-1431e58af4a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:00.557647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:00.557647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:00.557647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:00.557647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:00.557647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:00.557647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:00.557647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:00.557647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:00.557647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:00.557647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:00.557647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:00.557647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:00.557647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:00.557647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:00.557647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:00.558538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:00.558538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:00.558538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:00.558538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:00.558538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:00.558538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:00.558538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:00.558538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:00.558538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:00.558538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:00.558538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:00.558538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:00.558538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:00.558538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:00.558538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:00.558538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:00.559322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:00.559322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:00.559322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:00.559322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:00.559322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:00.559322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:00.559322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:00.559322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:00.559322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:00.559322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:00.559322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:00.559322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:00.559322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 65/129] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:00.560211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62167b-6854-41db-afd0-1431e58af4a0 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:00.583716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d6580-97c7-4ef2-bd33-c9e4f7230dab None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:00.627474 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557eefc2-ee35-48a6-93a4-66a031f977c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:00.628303 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557eefc2-ee35-48a6-93a4-66a031f977c4 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.628691 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557eefc2-ee35-48a6-93a4-66a031f977c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.628972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557eefc2-ee35-48a6-93a4-66a031f977c4 None admin] PATH_INFO: `/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.634152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557eefc2-ee35-48a6-93a4-66a031f977c4 None admin] RBAC: Authorizing `identity:get_user(user_id=ff6ae2c3a54f48faabac6aeb4ab6d81f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:00.635611 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557eefc2-ee35-48a6-93a4-66a031f977c4 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:00.643233 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 65/130] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:00.644075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557eefc2-ee35-48a6-93a4-66a031f977c4 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:00.777822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:00.783263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:00.802646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557eefc2-ee35-48a6-93a4-66a031f977c4 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:00.842561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b832c87-cc62-4245-a20e-122499163401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:00.842927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62167b-6854-41db-afd0-1431e58af4a0 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:00.843373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b832c87-cc62-4245-a20e-122499163401 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.843530 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b832c87-cc62-4245-a20e-122499163401 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.843777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b832c87-cc62-4245-a20e-122499163401 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.844572 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b832c87-cc62-4245-a20e-122499163401 None admin] RBAC: Adding query filter params (name=member) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:00.844990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b832c87-cc62-4245-a20e-122499163401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:00.846721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b832c87-cc62-4245-a20e-122499163401 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:00.856270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 66/131] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:00.857220 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b832c87-cc62-4245-a20e-122499163401 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:00.914568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a2e77-8483-40a1-b23c-0c52b9a08b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:00.917277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a2e77-8483-40a1-b23c-0c52b9a08b22 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.917572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a2e77-8483-40a1-b23c-0c52b9a08b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.917854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a2e77-8483-40a1-b23c-0c52b9a08b22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.918722 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a2e77-8483-40a1-b23c-0c52b9a08b22 None admin] RBAC: Adding query filter params (role.id=ba4641dd50c64f80a618b95aaaa321cb, scope.project.id=711e8b07480d478a80cbefc6fe30467f, user.id=717605caf5e34451b2fac90d3a2c249a) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:00.921394 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a2e77-8483-40a1-b23c-0c52b9a08b22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:00.922862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a2e77-8483-40a1-b23c-0c52b9a08b22 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:00.939317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 66/132] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/role_assignments?user.id=717605caf5e34451b2fac90d3a2c249a&scope.project.id=711e8b07480d478a80cbefc6fe30467f&role.id=ba4641dd50c64f80a618b95aaaa321cb => generated 259 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:00.940239 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a2e77-8483-40a1-b23c-0c52b9a08b22 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:01.089979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b832c87-cc62-4245-a20e-122499163401 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:01.130408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a2e77-8483-40a1-b23c-0c52b9a08b22 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:01.135456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdae651-452a-42cb-a67d-e0f4c864a359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:01.136812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdae651-452a-42cb-a67d-e0f4c864a359 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.137324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdae651-452a-42cb-a67d-e0f4c864a359 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.137828 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdae651-452a-42cb-a67d-e0f4c864a359 None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.139344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdae651-452a-42cb-a67d-e0f4c864a359 None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:01.141127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdae651-452a-42cb-a67d-e0f4c864a359 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:01.143168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 67/133] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:01.144561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52d2db-cb56-410f-a1d3-65bf813c005f None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.144958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52d2db-cb56-410f-a1d3-65bf813c005f None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.145507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52d2db-cb56-410f-a1d3-65bf813c005f None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.146943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 68/134] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:01.152431 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52d2db-cb56-410f-a1d3-65bf813c005f None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:01.169792 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:01.179375 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba195d-88d0-4cbd-b1fe-6dd0ca605329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:01.180522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba195d-88d0-4cbd-b1fe-6dd0ca605329 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.181020 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba195d-88d0-4cbd-b1fe-6dd0ca605329 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.181458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba195d-88d0-4cbd-b1fe-6dd0ca605329 None admin] PATH_INFO: `/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.184144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba195d-88d0-4cbd-b1fe-6dd0ca605329 None admin] RBAC: Authorizing `identity:get_user(user_id=ff6ae2c3a54f48faabac6aeb4ab6d81f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:01.187083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba195d-88d0-4cbd-b1fe-6dd0ca605329 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:01.190010 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 67/135] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:01.191918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83a344-0d62-4cdc-87f9-2ae6e65948c6 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.192387 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83a344-0d62-4cdc-87f9-2ae6e65948c6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.192859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83a344-0d62-4cdc-87f9-2ae6e65948c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.216034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ce83a344-0d62-4cdc-87f9-2ae6e65948c6 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:01.218371 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ce83a344-0d62-4cdc-87f9-2ae6e65948c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:01.234370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:01.323007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 68/136] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:01.329641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83a344-0d62-4cdc-87f9-2ae6e65948c6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:01.338277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52d2db-cb56-410f-a1d3-65bf813c005f None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:01.429772 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897f95f-cd54-4505-8254-9ebb46d6ffd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:01.431082 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897f95f-cd54-4505-8254-9ebb46d6ffd7 None admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.431208 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897f95f-cd54-4505-8254-9ebb46d6ffd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.431596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897f95f-cd54-4505-8254-9ebb46d6ffd7 None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f/users/ff6ae2c3a54f48faabac6aeb4ab6d81f/roles/abe200f23fc84ba397dd8c67874f0f2a` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.439713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7897f95f-cd54-4505-8254-9ebb46d6ffd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=711e8b07480d478a80cbefc6fe30467f, user_id=ff6ae2c3a54f48faabac6aeb4ab6d81f, role_id=abe200f23fc84ba397dd8c67874f0f2a)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:01.444514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7897f95f-cd54-4505-8254-9ebb46d6ffd7 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:01.468057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 69/137] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:01 2022] PUT /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f/users/ff6ae2c3a54f48faabac6aeb4ab6d81f/roles/abe200f23fc84ba397dd8c67874f0f2a => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:01.469121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897f95f-cd54-4505-8254-9ebb46d6ffd7 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:01.565249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83a344-0d62-4cdc-87f9-2ae6e65948c6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:01.601848 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:01.637134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e88e60-f92a-488c-996e-b450049803ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:01.638201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e88e60-f92a-488c-996e-b450049803ce None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.638596 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e88e60-f92a-488c-996e-b450049803ce None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.638972 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e88e60-f92a-488c-996e-b450049803ce None admin] PATH_INFO: `/v3/projects/249c9c15f2044b2891c272c99f0701d6` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.640604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e88e60-f92a-488c-996e-b450049803ce None admin] RBAC: Authorizing `identity:get_project(project_id=249c9c15f2044b2891c272c99f0701d6)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:01.642607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e88e60-f92a-488c-996e-b450049803ce None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:01.644618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 69/138] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:01 2022] GET /identity/v3/projects/249c9c15f2044b2891c272c99f0701d6 => generated 312 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:01.651577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e88e60-f92a-488c-996e-b450049803ce None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:01.666304 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897f95f-cd54-4505-8254-9ebb46d6ffd7 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:01.683549 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:01.758069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ac351-2c58-408d-9b94-e363689310b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:01.758899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ac351-2c58-408d-9b94-e363689310b2 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.759372 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ac351-2c58-408d-9b94-e363689310b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.759653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ac351-2c58-408d-9b94-e363689310b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.765646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ac351-2c58-408d-9b94-e363689310b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:01.767602 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ac351-2c58-408d-9b94-e363689310b2 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:01.769686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 70/139] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:01.775806 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ac351-2c58-408d-9b94-e363689310b2 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:01.886476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e88e60-f92a-488c-996e-b450049803ce None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:01.899219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ac351-2c58-408d-9b94-e363689310b2 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:01.955700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9ddbe-0403-40ec-a130-81d8d0579549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:01.956320 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9ddbe-0403-40ec-a130-81d8d0579549 None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.956650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9ddbe-0403-40ec-a130-81d8d0579549 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.956899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9ddbe-0403-40ec-a130-81d8d0579549 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.958110 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9ddbe-0403-40ec-a130-81d8d0579549 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:01.959717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9ddbe-0403-40ec-a130-81d8d0579549 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:01.976686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:01.983897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e394195-2883-40dd-9427-97f7548d8336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:01.984924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e394195-2883-40dd-9427-97f7548d8336 None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.985209 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e394195-2883-40dd-9427-97f7548d8336 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.985490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e394195-2883-40dd-9427-97f7548d8336 None admin] PATH_INFO: `/v3/projects/249c9c15f2044b2891c272c99f0701d6/users/ff6ae2c3a54f48faabac6aeb4ab6d81f/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.993707 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e394195-2883-40dd-9427-97f7548d8336 None admin] RBAC: Authorizing `identity:create_grant(project_id=249c9c15f2044b2891c272c99f0701d6, user_id=ff6ae2c3a54f48faabac6aeb4ab6d81f, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:01.994671 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 71/140] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:10:01 2022] POST /identity/v3/users => generated 283 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:10:01.995167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e394195-2883-40dd-9427-97f7548d8336 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:02.024080 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 70/141] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:01 2022] PUT /identity/v3/projects/249c9c15f2044b2891c272c99f0701d6/users/ff6ae2c3a54f48faabac6aeb4ab6d81f/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:02.565552 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb05a4-4459-4795-8939-994963946590 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:02.566111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb05a4-4459-4795-8939-994963946590 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:02.566111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb05a4-4459-4795-8939-994963946590 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:02.567623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 72/142] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:02.574040 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2020d8d-4062-4465-a5e2-11b655c40de3 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:02.574707 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2020d8d-4062-4465-a5e2-11b655c40de3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:02.575231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2020d8d-4062-4465-a5e2-11b655c40de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:02.595757 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c2020d8d-4062-4465-a5e2-11b655c40de3 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:02.597377 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c2020d8d-4062-4465-a5e2-11b655c40de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:02.685101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 71/143] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:02.716951 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb05a4-4459-4795-8939-994963946590 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:02.855958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb05a4-4459-4795-8939-994963946590 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:02.897899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c2e285-d510-4421-ab54-8903722756cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:02.899262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2e285-d510-4421-ab54-8903722756cc None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:02.899552 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2e285-d510-4421-ab54-8903722756cc None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:02.899845 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2e285-d510-4421-ab54-8903722756cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:02.917179 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c2e285-d510-4421-ab54-8903722756cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:02.924932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-c0c2e285-d510-4421-ab54-8903722756cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:02.924932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:02.924932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:02.924932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:02.924932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:02.924932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:02.924932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:02.924932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:02.924932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:02.924932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:02.924932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:02.924932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:02.924932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:02.924932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:02.924932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:02.924932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:02.925813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:02.925813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:02.925813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:02.925813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:02.925813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:02.925813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:02.925813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:02.925813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:02.925813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:02.925813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:02.925813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:02.925813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:02.925813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:02.925813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:02.925813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:02.925813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:02.926584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:02.926584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:02.926584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:02.926584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:02.926584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:02.926584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:02.926584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:02.926584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:02.926584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:02.926584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:02.926584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:02.926584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:02.926584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 73/144] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:02.934162 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2020d8d-4062-4465-a5e2-11b655c40de3 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:02.941739 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:02.974639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab441448-aef9-4bb3-b38e-cbd3852dcfac None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:02.974960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab441448-aef9-4bb3-b38e-cbd3852dcfac None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:02.975198 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab441448-aef9-4bb3-b38e-cbd3852dcfac None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:02.977267 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 74/145] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:02.988565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0688613-83a2-446e-a948-c5d4be413b45 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:02.988873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0688613-83a2-446e-a948-c5d4be413b45 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:02.989131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0688613-83a2-446e-a948-c5d4be413b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:03.016450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-d0688613-83a2-446e-a948-c5d4be413b45 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:03.017664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d0688613-83a2-446e-a948-c5d4be413b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:03.131654 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:03.136960 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2020d8d-4062-4465-a5e2-11b655c40de3 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:03.183473 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 75/146] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:03.217174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f8bc9-4f9f-4c21-a7ba-59df9c9bb3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:03.217801 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f8bc9-4f9f-4c21-a7ba-59df9c9bb3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:03.218102 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f8bc9-4f9f-4c21-a7ba-59df9c9bb3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:03.218380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f8bc9-4f9f-4c21-a7ba-59df9c9bb3a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:03.219153 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f8bc9-4f9f-4c21-a7ba-59df9c9bb3a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:03.219462 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f8bc9-4f9f-4c21-a7ba-59df9c9bb3a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:03.220899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f8bc9-4f9f-4c21-a7ba-59df9c9bb3a1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:03.230332 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 72/147] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:03.240349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0688613-83a2-446e-a948-c5d4be413b45 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:03.273810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f8bc9-4f9f-4c21-a7ba-59df9c9bb3a1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:03.380655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:03.440240 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0688613-83a2-446e-a948-c5d4be413b45 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:03.455216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f8bc9-4f9f-4c21-a7ba-59df9c9bb3a1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:03.512208 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f09839-35bb-4a5b-a5f3-9b2903ef0df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:03.513215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f09839-35bb-4a5b-a5f3-9b2903ef0df5 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:03.514303 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f09839-35bb-4a5b-a5f3-9b2903ef0df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:03.514691 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f09839-35bb-4a5b-a5f3-9b2903ef0df5 None admin] PATH_INFO: `/v3/users/44b032994ab44ba0acf66376eaab2451` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:03.516725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f09839-35bb-4a5b-a5f3-9b2903ef0df5 None admin] RBAC: Authorizing `identity:get_user(user_id=44b032994ab44ba0acf66376eaab2451)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:03.519745 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f09839-35bb-4a5b-a5f3-9b2903ef0df5 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:03.523229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 76/148] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:03 2022] GET /identity/v3/users/44b032994ab44ba0acf66376eaab2451 => generated 247 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:03.527370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c4733-3b81-4c17-844c-0bd6b0d7140d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:03.528318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c4733-3b81-4c17-844c-0bd6b0d7140d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:03.528737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c4733-3b81-4c17-844c-0bd6b0d7140d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:03.529108 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c4733-3b81-4c17-844c-0bd6b0d7140d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:03.532691 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f09839-35bb-4a5b-a5f3-9b2903ef0df5 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:03.557552 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c4733-3b81-4c17-844c-0bd6b0d7140d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:03.573307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-cc1c4733-3b81-4c17-844c-0bd6b0d7140d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:03.573307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:03.573307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:03.573307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:03.573307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:03.573307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:03.573307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:03.573307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:03.573307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:03.573307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:03.573307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:03.573307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:03.573307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:03.573307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:03.573307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:03.573307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:03.574207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:03.574207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:03.574207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:03.574207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:03.574207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:03.574207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:03.574207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:03.574207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:03.574207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:03.574207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:03.574207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:03.574207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:03.574207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:03.574207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:03.574207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:03.574207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:03.574954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:03.574954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:03.574954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:03.574954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:03.574954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:03.574954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:03.574954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:03.574954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:03.574954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:03.574954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:03.574954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:03.574954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:03.574954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 73/149] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:03.602338 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c4733-3b81-4c17-844c-0bd6b0d7140d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:03.635967 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:03.734334 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f09839-35bb-4a5b-a5f3-9b2903ef0df5 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:03.756401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c4733-3b81-4c17-844c-0bd6b0d7140d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:03.804349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d10f65-0ed8-4819-a631-bdb3c524a821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:03.806026 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d10f65-0ed8-4819-a631-bdb3c524a821 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:03.807611 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d10f65-0ed8-4819-a631-bdb3c524a821 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:03.808268 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d10f65-0ed8-4819-a631-bdb3c524a821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:03.825524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:03.842805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a985d-1fab-40b5-88cf-f42a10e1c50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:03.844834 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a985d-1fab-40b5-88cf-f42a10e1c50c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:03.845613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a985d-1fab-40b5-88cf-f42a10e1c50c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:03.847685 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a985d-1fab-40b5-88cf-f42a10e1c50c None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:03.848702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a985d-1fab-40b5-88cf-f42a10e1c50c None admin] RBAC: Adding query filter params (name=member) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:03.849222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a985d-1fab-40b5-88cf-f42a10e1c50c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:03.852195 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a985d-1fab-40b5-88cf-f42a10e1c50c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:03.870052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 74/150] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:03.876678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a985d-1fab-40b5-88cf-f42a10e1c50c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:03.920458 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d10f65-0ed8-4819-a631-bdb3c524a821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:03.921924 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d10f65-0ed8-4819-a631-bdb3c524a821 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:03.978497 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:04.031694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 77/151] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:04.038540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d10f65-0ed8-4819-a631-bdb3c524a821 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:04.122700 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a985d-1fab-40b5-88cf-f42a10e1c50c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:04.190284 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d10f65-0ed8-4819-a631-bdb3c524a821 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:04.206966 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f303ab2-7a52-4176-a9b0-7ad2a2855739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:04.208226 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f303ab2-7a52-4176-a9b0-7ad2a2855739 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.208734 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f303ab2-7a52-4176-a9b0-7ad2a2855739 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.209216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f303ab2-7a52-4176-a9b0-7ad2a2855739 None admin] PATH_INFO: `/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.211221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f303ab2-7a52-4176-a9b0-7ad2a2855739 None admin] RBAC: Authorizing `identity:get_user(user_id=ff6ae2c3a54f48faabac6aeb4ab6d81f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:04.212964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f303ab2-7a52-4176-a9b0-7ad2a2855739 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:04.220631 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 75/152] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:03 2022] GET /identity/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f => generated 275 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:04.222238 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd52372-b50d-4412-874c-da94f5dafcff None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.222532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd52372-b50d-4412-874c-da94f5dafcff None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.222914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd52372-b50d-4412-874c-da94f5dafcff None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.224294 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 76/153] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:04.227921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd52372-b50d-4412-874c-da94f5dafcff None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:04.228568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334e9e6-dd0b-4d76-8064-3b21c97d0dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:04.229597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334e9e6-dd0b-4d76-8064-3b21c97d0dcf None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.230017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334e9e6-dd0b-4d76-8064-3b21c97d0dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.230446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334e9e6-dd0b-4d76-8064-3b21c97d0dcf None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.231809 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334e9e6-dd0b-4d76-8064-3b21c97d0dcf None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:04.233571 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334e9e6-dd0b-4d76-8064-3b21c97d0dcf None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:04.235911 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 78/154] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:04 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:04.237360 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630d7b3-e4df-4a42-92bc-49da2ac7979b None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.237787 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630d7b3-e4df-4a42-92bc-49da2ac7979b None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.238432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630d7b3-e4df-4a42-92bc-49da2ac7979b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.263507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-3630d7b3-e4df-4a42-92bc-49da2ac7979b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:04.265760 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-3630d7b3-e4df-4a42-92bc-49da2ac7979b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:04.284116 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:04.341487 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:04.346640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 79/155] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:04.347758 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630d7b3-e4df-4a42-92bc-49da2ac7979b None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:04.463254 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd52372-b50d-4412-874c-da94f5dafcff None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:04.466002 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630d7b3-e4df-4a42-92bc-49da2ac7979b None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:04.514068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254066f-40ab-4567-b1a4-f9425ae86a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:04.515008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254066f-40ab-4567-b1a4-f9425ae86a7f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.515428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254066f-40ab-4567-b1a4-f9425ae86a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.515952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254066f-40ab-4567-b1a4-f9425ae86a7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.517134 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254066f-40ab-4567-b1a4-f9425ae86a7f None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=711e8b07480d478a80cbefc6fe30467f, user.id=44b032994ab44ba0acf66376eaab2451) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:04.517718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254066f-40ab-4567-b1a4-f9425ae86a7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:04.520015 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254066f-40ab-4567-b1a4-f9425ae86a7f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:04.527586 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 80/156] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:04 2022] GET /identity/v3/role_assignments?user.id=44b032994ab44ba0acf66376eaab2451&scope.project.id=711e8b07480d478a80cbefc6fe30467f&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:04.528631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254066f-40ab-4567-b1a4-f9425ae86a7f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:04.540963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d3d44-7d17-4cf3-be3c-9c2b78d61c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:04.541351 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d3d44-7d17-4cf3-be3c-9c2b78d61c9f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.541643 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d3d44-7d17-4cf3-be3c-9c2b78d61c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.541926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d3d44-7d17-4cf3-be3c-9c2b78d61c9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.547343 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:04.643174 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254066f-40ab-4567-b1a4-f9425ae86a7f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:04.673815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d3d44-7d17-4cf3-be3c-9c2b78d61c9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:04.675666 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d3d44-7d17-4cf3-be3c-9c2b78d61c9f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:04.682388 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee070fc3-338c-4e38-90ce-87b18cb44acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:04.683256 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee070fc3-338c-4e38-90ce-87b18cb44acc None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.683517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee070fc3-338c-4e38-90ce-87b18cb44acc None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.683791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee070fc3-338c-4e38-90ce-87b18cb44acc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.696919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee070fc3-338c-4e38-90ce-87b18cb44acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:04.705229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-ee070fc3-338c-4e38-90ce-87b18cb44acc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:04.705229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:04.705229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:04.705229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:04.705229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:04.705229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:04.705229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:04.705229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:04.705229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:04.705229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:04.705229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:04.705229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:04.705229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:04.705229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:04.705229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:04.705229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:04.706416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:04.706416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:04.706416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:04.706416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:04.706416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:04.706416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:04.706416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:04.706416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:04.706416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:04.706416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:04.706416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:04.706416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:04.706416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:04.706416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:04.706416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:04.706416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:04.707344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:04.707344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:04.707344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:04.707344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:04.707344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:04.707344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:04.707344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:04.707344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:04.707344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:04.707344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:04.707344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:04.707344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:04.707344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 81/157] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:04.715193 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee070fc3-338c-4e38-90ce-87b18cb44acc None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:04.785649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:04.800649 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:04.829800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee070fc3-338c-4e38-90ce-87b18cb44acc None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:04.868356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5eab85-ece4-4c79-a458-573cd1b2cc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:04.869396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5eab85-ece4-4c79-a458-573cd1b2cc40 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.869592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 77/158] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:10:04.870058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5eab85-ece4-4c79-a458-573cd1b2cc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.870534 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5eab85-ece4-4c79-a458-573cd1b2cc40 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.871491 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5eab85-ece4-4c79-a458-573cd1b2cc40 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:04.871996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5eab85-ece4-4c79-a458-573cd1b2cc40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:04.873882 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5eab85-ece4-4c79-a458-573cd1b2cc40 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:04.874693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d3d44-7d17-4cf3-be3c-9c2b78d61c9f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:04.888626 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 82/159] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:04.898783 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5eab85-ece4-4c79-a458-573cd1b2cc40 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:05.099187 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5eab85-ece4-4c79-a458-573cd1b2cc40 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:05.181125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e9e0d-c99e-4d8f-bf0e-ad24a2768280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:05.182393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e9e0d-c99e-4d8f-bf0e-ad24a2768280 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.184275 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e9e0d-c99e-4d8f-bf0e-ad24a2768280 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.185041 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e9e0d-c99e-4d8f-bf0e-ad24a2768280 None admin] PATH_INFO: `/v3/users/717605caf5e34451b2fac90d3a2c249a` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.192740 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e9e0d-c99e-4d8f-bf0e-ad24a2768280 None admin] RBAC: Authorizing `identity:get_user(user_id=717605caf5e34451b2fac90d3a2c249a)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:05.194574 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e9e0d-c99e-4d8f-bf0e-ad24a2768280 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:05.197170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 83/160] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:04 2022] GET /identity/v3/users/717605caf5e34451b2fac90d3a2c249a => generated 289 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:05.206341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b43f7a-72ca-4228-906f-7777f1bb7ada None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.206848 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b43f7a-72ca-4228-906f-7777f1bb7ada None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.207346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b43f7a-72ca-4228-906f-7777f1bb7ada None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.209330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 84/161] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:05.213181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b43f7a-72ca-4228-906f-7777f1bb7ada None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:05.227858 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:05.240666 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d3d44-7d17-4cf3-be3c-9c2b78d61c9f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:05.349272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22291281-9c59-4664-8db3-2113528e2f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:05.350090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22291281-9c59-4664-8db3-2113528e2f75 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.350390 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22291281-9c59-4664-8db3-2113528e2f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.350670 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22291281-9c59-4664-8db3-2113528e2f75 None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.352122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22291281-9c59-4664-8db3-2113528e2f75 None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:05.353689 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22291281-9c59-4664-8db3-2113528e2f75 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:05.355557 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 78/162] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:04 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 482 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:05.364392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362178d1-58db-4621-a3a2-c8f0fc45fb2c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.364681 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362178d1-58db-4621-a3a2-c8f0fc45fb2c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.365026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362178d1-58db-4621-a3a2-c8f0fc45fb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.402558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-362178d1-58db-4621-a3a2-c8f0fc45fb2c None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:05.403799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-362178d1-58db-4621-a3a2-c8f0fc45fb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:05.437730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b43f7a-72ca-4228-906f-7777f1bb7ada None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:05.453221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:05.522149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648d14f-891a-4fac-9cc3-41d0c384c8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:05.525910 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648d14f-891a-4fac-9cc3-41d0c384c8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.528995 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648d14f-891a-4fac-9cc3-41d0c384c8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.531333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648d14f-891a-4fac-9cc3-41d0c384c8c2 None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.535894 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648d14f-891a-4fac-9cc3-41d0c384c8c2 None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:05.545605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648d14f-891a-4fac-9cc3-41d0c384c8c2 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:05.551876 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 85/163] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:05 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:05.555966 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 79/164] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:05.556842 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362178d1-58db-4621-a3a2-c8f0fc45fb2c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:05.562175 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648d14f-891a-4fac-9cc3-41d0c384c8c2 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:05.734022 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362178d1-58db-4621-a3a2-c8f0fc45fb2c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:05.826096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:05.833044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3292fd-e3b3-489f-ae33-9ee98ca2e6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:05.833736 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3292fd-e3b3-489f-ae33-9ee98ca2e6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.834025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3292fd-e3b3-489f-ae33-9ee98ca2e6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.834308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3292fd-e3b3-489f-ae33-9ee98ca2e6b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.835175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3292fd-e3b3-489f-ae33-9ee98ca2e6b1 None admin] RBAC: Adding query filter params (role.id=18f0dc41b78c4b52b6912225928ea388, scope.project.id=711e8b07480d478a80cbefc6fe30467f, user.id=ff6ae2c3a54f48faabac6aeb4ab6d81f) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:05.835531 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3292fd-e3b3-489f-ae33-9ee98ca2e6b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:05.837500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3292fd-e3b3-489f-ae33-9ee98ca2e6b1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:05.859846 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 80/165] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:05 2022] GET /identity/v3/role_assignments?user.id=ff6ae2c3a54f48faabac6aeb4ab6d81f&scope.project.id=711e8b07480d478a80cbefc6fe30467f&role.id=18f0dc41b78c4b52b6912225928ea388 => generated 608 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:05.866016 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3292fd-e3b3-489f-ae33-9ee98ca2e6b1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:05.950482 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648d14f-891a-4fac-9cc3-41d0c384c8c2 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:05.956639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:06.011872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526e34c-b41e-4dc1-bd81-a848fed9e38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:06.012743 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526e34c-b41e-4dc1-bd81-a848fed9e38a None admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.013029 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526e34c-b41e-4dc1-bd81-a848fed9e38a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.013304 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526e34c-b41e-4dc1-bd81-a848fed9e38a None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f/users/717605caf5e34451b2fac90d3a2c249a/roles/ba4641dd50c64f80a618b95aaaa321cb` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.015380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526e34c-b41e-4dc1-bd81-a848fed9e38a None admin] RBAC: Authorizing `identity:create_grant(project_id=711e8b07480d478a80cbefc6fe30467f, user_id=717605caf5e34451b2fac90d3a2c249a, role_id=ba4641dd50c64f80a618b95aaaa321cb)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:06.016900 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526e34c-b41e-4dc1-bd81-a848fed9e38a None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:06.037653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 86/166] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:05 2022] PUT /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f/users/717605caf5e34451b2fac90d3a2c249a/roles/ba4641dd50c64f80a618b95aaaa321cb => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:06.128508 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3292fd-e3b3-489f-ae33-9ee98ca2e6b1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:06.148028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be336262-0763-40d5-8878-c453f28389e0 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.148425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be336262-0763-40d5-8878-c453f28389e0 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.148622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be336262-0763-40d5-8878-c453f28389e0 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.150318 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 87/167] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:06.157233 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ada2a8-9145-48dd-8d2f-df1b189cf36e None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.157551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ada2a8-9145-48dd-8d2f-df1b189cf36e None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.157805 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ada2a8-9145-48dd-8d2f-df1b189cf36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.182247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2223cfb-4b38-44d7-af2a-4a16c7c4e41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:06.183073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2223cfb-4b38-44d7-af2a-4a16c7c4e41e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.183357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2223cfb-4b38-44d7-af2a-4a16c7c4e41e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.183632 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2223cfb-4b38-44d7-af2a-4a16c7c4e41e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.186049 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-e6ada2a8-9145-48dd-8d2f-df1b189cf36e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:06.187287 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e6ada2a8-9145-48dd-8d2f-df1b189cf36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:06.213521 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2223cfb-4b38-44d7-af2a-4a16c7c4e41e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:06.221910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-f2223cfb-4b38-44d7-af2a-4a16c7c4e41e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:06.221910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:06.221910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:06.221910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:06.221910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:06.221910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:06.221910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:06.221910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:06.221910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:06.221910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:06.221910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:06.221910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:06.221910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:06.221910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:06.221910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:06.221910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:06.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:06.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:06.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:06.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:06.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:06.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:06.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:06.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:06.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:06.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:06.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:06.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:06.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:06.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:06.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:06.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:06.223749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:06.223749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:06.223749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:06.223749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:06.223749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:06.223749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:06.223749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:06.223749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:06.223749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:06.223749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:06.223749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:06.223749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:06.223749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 81/168] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:06.233394 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2223cfb-4b38-44d7-af2a-4a16c7c4e41e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:06.327442 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 88/169] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:06.375137 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ada2a8-9145-48dd-8d2f-df1b189cf36e None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:06.408250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:06.450776 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2223cfb-4b38-44d7-af2a-4a16c7c4e41e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:06.480940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:06.515031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d8fab-a70d-4df7-9b4e-3b0014556220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:06.515712 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d8fab-a70d-4df7-9b4e-3b0014556220 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.515996 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d8fab-a70d-4df7-9b4e-3b0014556220 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.516270 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d8fab-a70d-4df7-9b4e-3b0014556220 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.517111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d8fab-a70d-4df7-9b4e-3b0014556220 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:06.517421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d8fab-a70d-4df7-9b4e-3b0014556220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:06.518810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d8fab-a70d-4df7-9b4e-3b0014556220 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:06.530704 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 82/170] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:06.537308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d8fab-a70d-4df7-9b4e-3b0014556220 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:06.546921 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ada2a8-9145-48dd-8d2f-df1b189cf36e None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:06.619636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89496de2-f7b8-4b9a-b70e-08f6ea55f848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:06.629182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89496de2-f7b8-4b9a-b70e-08f6ea55f848 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.629823 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89496de2-f7b8-4b9a-b70e-08f6ea55f848 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.630335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89496de2-f7b8-4b9a-b70e-08f6ea55f848 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.652642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89496de2-f7b8-4b9a-b70e-08f6ea55f848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:06.664106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-89496de2-f7b8-4b9a-b70e-08f6ea55f848 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:06.664106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:06.664106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:06.664106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:06.664106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:06.664106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:06.664106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:06.664106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:06.664106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:06.664106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:06.664106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:06.664106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:06.664106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:06.664106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:06.664106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:06.664106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:06.665115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:06.665115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:06.665115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:06.665115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:06.665115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:06.665115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:06.665115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:06.665115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:06.665115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:06.665115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:06.665115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:06.665115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:06.665115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:06.665115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:06.665115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:06.665115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:06.665964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:06.665964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:06.665964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:06.665964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:06.665964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:06.665964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:06.665964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:06.665964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:06.665964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:06.665964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:06.665964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:06.665964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:06.668073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 89/171] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:06.674264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89496de2-f7b8-4b9a-b70e-08f6ea55f848 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:06.756416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d8fab-a70d-4df7-9b4e-3b0014556220 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:06.839785 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3041765-a710-4f3e-a77b-fe32ccb5e121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:06.841134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3041765-a710-4f3e-a77b-fe32ccb5e121 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.841591 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3041765-a710-4f3e-a77b-fe32ccb5e121 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.842021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3041765-a710-4f3e-a77b-fe32ccb5e121 None admin] PATH_INFO: `/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.843496 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3041765-a710-4f3e-a77b-fe32ccb5e121 None admin] RBAC: Authorizing `identity:get_user(user_id=ff6ae2c3a54f48faabac6aeb4ab6d81f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:06.845422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3041765-a710-4f3e-a77b-fe32ccb5e121 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:06.847371 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 83/172] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:06 2022] GET /identity/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f => generated 275 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:06.854567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3041765-a710-4f3e-a77b-fe32ccb5e121 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:06.872394 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89496de2-f7b8-4b9a-b70e-08f6ea55f848 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:06.876171 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:06.921926 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:06.946827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1aba7-8ca7-45bf-ad65-e366f437ee6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:06.947526 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1aba7-8ca7-45bf-ad65-e366f437ee6e None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.947815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1aba7-8ca7-45bf-ad65-e366f437ee6e None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.948109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1aba7-8ca7-45bf-ad65-e366f437ee6e None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.948886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1aba7-8ca7-45bf-ad65-e366f437ee6e None admin] RBAC: Adding query filter params (name=member) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:06.949197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1aba7-8ca7-45bf-ad65-e366f437ee6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:06.950589 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1aba7-8ca7-45bf-ad65-e366f437ee6e None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:06.966981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 90/173] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:06.968371 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c9eb0-a70f-4e9a-9859-ca64e2c7e811 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.972930 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c9eb0-a70f-4e9a-9859-ca64e2c7e811 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.972930 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c9eb0-a70f-4e9a-9859-ca64e2c7e811 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.972930 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 91/174] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:06.973977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c9eb0-a70f-4e9a-9859-ca64e2c7e811 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:06.994156 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3041765-a710-4f3e-a77b-fe32ccb5e121 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:07.065511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936caa92-e258-4dba-90b5-b8e13d7f25a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:07.066176 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936caa92-e258-4dba-90b5-b8e13d7f25a5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:07.066459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936caa92-e258-4dba-90b5-b8e13d7f25a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:07.066735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936caa92-e258-4dba-90b5-b8e13d7f25a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:07.145542 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c9eb0-a70f-4e9a-9859-ca64e2c7e811 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:07.189083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2122d-b324-4c56-98eb-eb0a55620d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:07.189975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2122d-b324-4c56-98eb-eb0a55620d24 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:07.190246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2122d-b324-4c56-98eb-eb0a55620d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:07.190526 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2122d-b324-4c56-98eb-eb0a55620d24 None admin] PATH_INFO: `/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:07.191759 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2122d-b324-4c56-98eb-eb0a55620d24 None admin] RBAC: Authorizing `identity:get_user(user_id=ff6ae2c3a54f48faabac6aeb4ab6d81f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:07.193245 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2122d-b324-4c56-98eb-eb0a55620d24 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:07.195451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 92/175] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:06 2022] GET /identity/v3/users/ff6ae2c3a54f48faabac6aeb4ab6d81f => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:07.196763 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c1d8c-725e-4ad5-82ce-2bc0c8c5a098 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:07.196964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c1d8c-725e-4ad5-82ce-2bc0c8c5a098 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:07.197630 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c1d8c-725e-4ad5-82ce-2bc0c8c5a098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:07.215503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-9a9c1d8c-725e-4ad5-82ce-2bc0c8c5a098 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:07.216833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-9a9c1d8c-725e-4ad5-82ce-2bc0c8c5a098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:07.241458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936caa92-e258-4dba-90b5-b8e13d7f25a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:07.243513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936caa92-e258-4dba-90b5-b8e13d7f25a5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:07.270041 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:07.271016 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:07.287720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 93/176] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:07.288418 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c1d8c-725e-4ad5-82ce-2bc0c8c5a098 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:07.443352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 84/177] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:07.445742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936caa92-e258-4dba-90b5-b8e13d7f25a5 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:07.463834 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c1d8c-725e-4ad5-82ce-2bc0c8c5a098 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:07.500221 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65a41b-cca2-47b3-b984-b2b462675ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:07.500917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65a41b-cca2-47b3-b984-b2b462675ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:07.501205 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65a41b-cca2-47b3-b984-b2b462675ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:07.501481 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65a41b-cca2-47b3-b984-b2b462675ba4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:07.566947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:07.590509 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65a41b-cca2-47b3-b984-b2b462675ba4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:07.591994 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65a41b-cca2-47b3-b984-b2b462675ba4 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:07.635251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936caa92-e258-4dba-90b5-b8e13d7f25a5 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:07.652825 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:07.697251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79ce06-13d0-485d-b07d-2d9144c14624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:07.698385 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79ce06-13d0-485d-b07d-2d9144c14624 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:07.698856 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79ce06-13d0-485d-b07d-2d9144c14624 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:07.699325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79ce06-13d0-485d-b07d-2d9144c14624 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:07.709984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 94/178] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:07.711426 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d8635-da96-4674-9960-66c70cfb3c1d None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:07.711791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d8635-da96-4674-9960-66c70cfb3c1d None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:07.712137 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d8635-da96-4674-9960-66c70cfb3c1d None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:07.713646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 95/179] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:07.714362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d8635-da96-4674-9960-66c70cfb3c1d None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:07.724992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79ce06-13d0-485d-b07d-2d9144c14624 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:07.733234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-bc79ce06-13d0-485d-b07d-2d9144c14624 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:07.733234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:07.733234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:07.733234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:07.733234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:07.733234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:07.733234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:07.733234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:07.733234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:07.733234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:07.733234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:07.733234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:07.733234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:07.733234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:07.733234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:07.733234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:07.734308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:07.734308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:07.734308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:07.734308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:07.734308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:07.734308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:07.734308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:07.734308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:07.734308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:07.734308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:07.734308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:07.734308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:07.734308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:07.734308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:07.734308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:07.734308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:07.735285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:07.735285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:07.735285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:07.735285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:07.735285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:07.735285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:07.735285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:07.735285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:07.735285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:07.735285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:07.735285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:07.735285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:07.735285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 85/180] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:07.736078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f72a4b-3bac-4814-a578-28e08ef3a2c3 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:07.736078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f72a4b-3bac-4814-a578-28e08ef3a2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:07.736215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f72a4b-3bac-4814-a578-28e08ef3a2c3 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:07.741660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 86/181] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:07.741660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f72a4b-3bac-4814-a578-28e08ef3a2c3 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:07.815879 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:07.839623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d8635-da96-4674-9960-66c70cfb3c1d None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:07.856491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f72a4b-3bac-4814-a578-28e08ef3a2c3 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:07.878160 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99d4b5-acad-4ca9-baad-8c32a224c51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:07.879399 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99d4b5-acad-4ca9-baad-8c32a224c51d None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:07.879399 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99d4b5-acad-4ca9-baad-8c32a224c51d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:07.879766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99d4b5-acad-4ca9-baad-8c32a224c51d None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:07.881003 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99d4b5-acad-4ca9-baad-8c32a224c51d None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:07.882404 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99d4b5-acad-4ca9-baad-8c32a224c51d None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:07.884280 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 96/182] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:07 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:07.885289 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255210bd-2142-446a-89e9-4e8efff1542e None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:07.885746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255210bd-2142-446a-89e9-4e8efff1542e None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:07.886077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255210bd-2142-446a-89e9-4e8efff1542e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:07.902292 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0763c-60db-441d-b0fe-1e90208c83b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:07.903158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0763c-60db-441d-b0fe-1e90208c83b4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:07.903444 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0763c-60db-441d-b0fe-1e90208c83b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:07.903789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0763c-60db-441d-b0fe-1e90208c83b4 None admin] PATH_INFO: `/v3/projects/249c9c15f2044b2891c272c99f0701d6` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:07.903904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-255210bd-2142-446a-89e9-4e8efff1542e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:07.904960 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0763c-60db-441d-b0fe-1e90208c83b4 None admin] RBAC: Authorizing `identity:get_project(project_id=249c9c15f2044b2891c272c99f0701d6)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:07.905073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-255210bd-2142-446a-89e9-4e8efff1542e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:07.906849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0763c-60db-441d-b0fe-1e90208c83b4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:07.908157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 87/183] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:07 2022] GET /identity/v3/projects/249c9c15f2044b2891c272c99f0701d6 => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:07.914386 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0763c-60db-441d-b0fe-1e90208c83b4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:07.967758 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:07.986978 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 97/184] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:07.993653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3064a-4e75-4e6b-8ad3-5816308b68d6 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:07.994932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3064a-4e75-4e6b-8ad3-5816308b68d6 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:07.995434 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3064a-4e75-4e6b-8ad3-5816308b68d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.017193 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-0ed3064a-4e75-4e6b-8ad3-5816308b68d6 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:08.019456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0ed3064a-4e75-4e6b-8ad3-5816308b68d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:08.051120 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0763c-60db-441d-b0fe-1e90208c83b4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:08.088395 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e87d4c-3abb-45a8-bf37-e6b11bcf30d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:08.089231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e87d4c-3abb-45a8-bf37-e6b11bcf30d6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.089613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e87d4c-3abb-45a8-bf37-e6b11bcf30d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.089989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e87d4c-3abb-45a8-bf37-e6b11bcf30d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.090849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e87d4c-3abb-45a8-bf37-e6b11bcf30d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:08.091262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e87d4c-3abb-45a8-bf37-e6b11bcf30d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:08.092784 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e87d4c-3abb-45a8-bf37-e6b11bcf30d6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:08.106471 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 88/185] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:08.106471 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e87d4c-3abb-45a8-bf37-e6b11bcf30d6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:08.111853 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 98/186] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:08.112678 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3064a-4e75-4e6b-8ad3-5816308b68d6 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:08.130376 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:08.227634 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:08.231598 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e87d4c-3abb-45a8-bf37-e6b11bcf30d6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:08.235330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3064a-4e75-4e6b-8ad3-5816308b68d6 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:08.272100 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa72df5-2cb6-450e-8796-0f001f3ae8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:08.273442 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa72df5-2cb6-450e-8796-0f001f3ae8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.273792 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa72df5-2cb6-450e-8796-0f001f3ae8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.274165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa72df5-2cb6-450e-8796-0f001f3ae8e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.274955 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355722f0-ed86-468e-8918-61d8921e0fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:08.275155 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa72df5-2cb6-450e-8796-0f001f3ae8e7 None admin] RBAC: Adding query filter params (role.id=18f0dc41b78c4b52b6912225928ea388, scope.project.id=249c9c15f2044b2891c272c99f0701d6, user.id=ff6ae2c3a54f48faabac6aeb4ab6d81f) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:08.275533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa72df5-2cb6-450e-8796-0f001f3ae8e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:08.275804 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355722f0-ed86-468e-8918-61d8921e0fca None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.276153 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355722f0-ed86-468e-8918-61d8921e0fca None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.276522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355722f0-ed86-468e-8918-61d8921e0fca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.277259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa72df5-2cb6-450e-8796-0f001f3ae8e7 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:08.277598 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355722f0-ed86-468e-8918-61d8921e0fca None admin] RBAC: Adding query filter params (role.id=abe200f23fc84ba397dd8c67874f0f2a, scope.project.id=711e8b07480d478a80cbefc6fe30467f, user.id=ff6ae2c3a54f48faabac6aeb4ab6d81f) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:08.277973 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355722f0-ed86-468e-8918-61d8921e0fca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:08.279463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355722f0-ed86-468e-8918-61d8921e0fca None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:08.292684 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 99/187] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:08 2022] GET /identity/v3/role_assignments?user.id=ff6ae2c3a54f48faabac6aeb4ab6d81f&scope.project.id=249c9c15f2044b2891c272c99f0701d6&role.id=18f0dc41b78c4b52b6912225928ea388 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:08.292684 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa72df5-2cb6-450e-8796-0f001f3ae8e7 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:08.292918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 89/188] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:08 2022] GET /identity/v3/role_assignments?user.id=ff6ae2c3a54f48faabac6aeb4ab6d81f&scope.project.id=711e8b07480d478a80cbefc6fe30467f&role.id=abe200f23fc84ba397dd8c67874f0f2a => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:08.292918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355722f0-ed86-468e-8918-61d8921e0fca None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:08.470166 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:08.476627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355722f0-ed86-468e-8918-61d8921e0fca None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:08.525506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa72df5-2cb6-450e-8796-0f001f3ae8e7 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:08.527718 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddcc28-7e2f-4f85-ad4d-05d7680f0bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:08.528888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddcc28-7e2f-4f85-ad4d-05d7680f0bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.529381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddcc28-7e2f-4f85-ad4d-05d7680f0bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.529849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddcc28-7e2f-4f85-ad4d-05d7680f0bf7 None admin] PATH_INFO: `/v3/users/ae2439475b714dd1ba40dbe86303d708` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.547926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddcc28-7e2f-4f85-ad4d-05d7680f0bf7 None admin] RBAC: Authorizing `identity:get_user(user_id=ae2439475b714dd1ba40dbe86303d708)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:08.549837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddcc28-7e2f-4f85-ad4d-05d7680f0bf7 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:08.552088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 90/189] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:08 2022] GET /identity/v3/users/ae2439475b714dd1ba40dbe86303d708 => generated 283 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:08.553225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddcc28-7e2f-4f85-ad4d-05d7680f0bf7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:08.567223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87d91a-a669-4d65-9415-b1a3f2e8db64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:08.568318 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87d91a-a669-4d65-9415-b1a3f2e8db64 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.568626 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87d91a-a669-4d65-9415-b1a3f2e8db64 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.569003 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87d91a-a669-4d65-9415-b1a3f2e8db64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.580207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:08.588606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87d91a-a669-4d65-9415-b1a3f2e8db64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:08.597010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-9b87d91a-a669-4d65-9415-b1a3f2e8db64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:08.597010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:08.597010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:08.597010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:08.597010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:08.597010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:08.597010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:08.597010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:08.597010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:08.597010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:08.597010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:08.597010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:08.597010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:08.597010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:08.597010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:08.597010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:08.598120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:08.598120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:08.598120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:08.598120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:08.598120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:08.598120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:08.598120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:08.598120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:08.598120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:08.598120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:08.598120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:08.598120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:08.598120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:08.598120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:08.598120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:08.598120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:08.599145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:08.599145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:08.599145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:08.599145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:08.599145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:08.599145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:08.599145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:08.599145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:08.599145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:08.599145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:08.599145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:08.599145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:08.599145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 100/190] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:08.603110 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87d91a-a669-4d65-9415-b1a3f2e8db64 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:08.700360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddcc28-7e2f-4f85-ad4d-05d7680f0bf7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:08.730269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87d91a-a669-4d65-9415-b1a3f2e8db64 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:08.734871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6d0ec-e811-4775-838f-62baaae23ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:08.735666 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6d0ec-e811-4775-838f-62baaae23ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.735946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6d0ec-e811-4775-838f-62baaae23ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.736219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6d0ec-e811-4775-838f-62baaae23ab1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.737416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6d0ec-e811-4775-838f-62baaae23ab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:08.738857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6d0ec-e811-4775-838f-62baaae23ab1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:08.740716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 91/191] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:08.741269 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6d0ec-e811-4775-838f-62baaae23ab1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:08.767215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8ef77-5862-4fdc-a0ab-fde263bfe311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:08.767868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8ef77-5862-4fdc-a0ab-fde263bfe311 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.768150 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8ef77-5862-4fdc-a0ab-fde263bfe311 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.768485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8ef77-5862-4fdc-a0ab-fde263bfe311 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.771225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:08.829603 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:08.842119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8ef77-5862-4fdc-a0ab-fde263bfe311 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:08.842484 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6d0ec-e811-4775-838f-62baaae23ab1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:08.843594 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8ef77-5862-4fdc-a0ab-fde263bfe311 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:08.877220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6296cb2-a09a-44b7-8ee5-adbb88ece15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:08.878002 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6296cb2-a09a-44b7-8ee5-adbb88ece15f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.878292 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6296cb2-a09a-44b7-8ee5-adbb88ece15f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.878574 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6296cb2-a09a-44b7-8ee5-adbb88ece15f None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.879327 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6296cb2-a09a-44b7-8ee5-adbb88ece15f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:08.879635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6296cb2-a09a-44b7-8ee5-adbb88ece15f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:08.881058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6296cb2-a09a-44b7-8ee5-adbb88ece15f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:08.889665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 92/192] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:08.891081 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e963f0-24d9-4c2d-bcbc-98eb91b2fa8f None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.891375 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e963f0-24d9-4c2d-bcbc-98eb91b2fa8f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.891660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e963f0-24d9-4c2d-bcbc-98eb91b2fa8f None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.897113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 93/193] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:08.898383 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e963f0-24d9-4c2d-bcbc-98eb91b2fa8f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:08.944259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 101/194] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:08.945816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8ef77-5862-4fdc-a0ab-fde263bfe311 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:08.978854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:09.002267 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e963f0-24d9-4c2d-bcbc-98eb91b2fa8f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.036773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7679254-ccf4-4303-9138-d206965998c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.037496 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7679254-ccf4-4303-9138-d206965998c0 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.037826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7679254-ccf4-4303-9138-d206965998c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.038145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7679254-ccf4-4303-9138-d206965998c0 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.038964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7679254-ccf4-4303-9138-d206965998c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:09.040634 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7679254-ccf4-4303-9138-d206965998c0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:09.045834 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8ef77-5862-4fdc-a0ab-fde263bfe311 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.059181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 94/195] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:10:08 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:09.060999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3014d90-65c0-453b-9ed8-eac35f7e0469 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.061293 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3014d90-65c0-453b-9ed8-eac35f7e0469 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.061580 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3014d90-65c0-453b-9ed8-eac35f7e0469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.084034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:09.091184 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e3014d90-65c0-453b-9ed8-eac35f7e0469 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:09.093910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e3014d90-65c0-453b-9ed8-eac35f7e0469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:09.095082 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489a19f-2974-4db0-9697-00af55bd4d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.095992 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489a19f-2974-4db0-9697-00af55bd4d2f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.096523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489a19f-2974-4db0-9697-00af55bd4d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.096814 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489a19f-2974-4db0-9697-00af55bd4d2f None admin] PATH_INFO: `/v3/users/ae2439475b714dd1ba40dbe86303d708` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.099074 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489a19f-2974-4db0-9697-00af55bd4d2f None admin] RBAC: Authorizing `identity:get_user(user_id=ae2439475b714dd1ba40dbe86303d708)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:09.100585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489a19f-2974-4db0-9697-00af55bd4d2f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:09.102370 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 102/196] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:08 2022] GET /identity/v3/users/ae2439475b714dd1ba40dbe86303d708 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:09.103147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489a19f-2974-4db0-9697-00af55bd4d2f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.171760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 95/197] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:09.172983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3014d90-65c0-453b-9ed8-eac35f7e0469 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.236324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489a19f-2974-4db0-9697-00af55bd4d2f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.238367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:09.281144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4e4af-20f6-454e-8aa7-826f5b851cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.282323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4e4af-20f6-454e-8aa7-826f5b851cfe None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.282804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4e4af-20f6-454e-8aa7-826f5b851cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.283088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4e4af-20f6-454e-8aa7-826f5b851cfe None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.284429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4e4af-20f6-454e-8aa7-826f5b851cfe None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:09.285876 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4e4af-20f6-454e-8aa7-826f5b851cfe None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:09.287599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 103/198] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:09.288474 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4e4af-20f6-454e-8aa7-826f5b851cfe None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.305719 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3014d90-65c0-453b-9ed8-eac35f7e0469 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.359269 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf882ba-fe8c-4f64-87ca-9a1182d1b7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.359933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf882ba-fe8c-4f64-87ca-9a1182d1b7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.360218 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf882ba-fe8c-4f64-87ca-9a1182d1b7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.360536 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf882ba-fe8c-4f64-87ca-9a1182d1b7b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.366051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:09.440904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4e4af-20f6-454e-8aa7-826f5b851cfe None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.442317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf882ba-fe8c-4f64-87ca-9a1182d1b7b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:09.443795 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf882ba-fe8c-4f64-87ca-9a1182d1b7b7 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:09.493865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647667f-8e84-4f76-8914-1c1ec0bc810c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.495745 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647667f-8e84-4f76-8914-1c1ec0bc810c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.496240 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647667f-8e84-4f76-8914-1c1ec0bc810c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.496774 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647667f-8e84-4f76-8914-1c1ec0bc810c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.513559 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d647667f-8e84-4f76-8914-1c1ec0bc810c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:09.525339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-d647667f-8e84-4f76-8914-1c1ec0bc810c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:09.525339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:09.525339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:09.525339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:09.525339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:09.525339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:09.525339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:09.525339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:09.525339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:09.525339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:09.525339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:09.525339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:09.525339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:09.525339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:09.525339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:09.525339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:09.526252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:09.526252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:09.526252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:09.526252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:09.526252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:09.526252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:09.526252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:09.526252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:09.526252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:09.526252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:09.526252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:09.526252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:09.526252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:09.526252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:09.526252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:09.526252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:09.527036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:09.527036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:09.527036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:09.527036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:09.527036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:09.527036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:09.527036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:09.527036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:09.527036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:09.527036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:09.527036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:09.527036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:09.529659 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 104/199] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:09.530469 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647667f-8e84-4f76-8914-1c1ec0bc810c None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.563963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 96/200] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:09.564964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf882ba-fe8c-4f64-87ca-9a1182d1b7b7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.574280 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:09.601218 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:09.655395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647667f-8e84-4f76-8914-1c1ec0bc810c None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.669029 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf882ba-fe8c-4f64-87ca-9a1182d1b7b7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.697068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f9cb3d-d3c9-4c87-8f1b-4d891126d5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.698171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9cb3d-d3c9-4c87-8f1b-4d891126d5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.698662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9cb3d-d3c9-4c87-8f1b-4d891126d5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.699155 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9cb3d-d3c9-4c87-8f1b-4d891126d5c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.700434 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f9cb3d-d3c9-4c87-8f1b-4d891126d5c6 None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=359a065a7d2c49fea870a82587a5d51e, user.id=ae2439475b714dd1ba40dbe86303d708) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:09.700963 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f9cb3d-d3c9-4c87-8f1b-4d891126d5c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:09.702724 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f9cb3d-d3c9-4c87-8f1b-4d891126d5c6 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:09.705403 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52495ba8-7799-4831-ac11-34d4cbd5ee3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.706076 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495ba8-7799-4831-ac11-34d4cbd5ee3c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.706357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495ba8-7799-4831-ac11-34d4cbd5ee3c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.706632 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495ba8-7799-4831-ac11-34d4cbd5ee3c None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.707438 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52495ba8-7799-4831-ac11-34d4cbd5ee3c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:09.707683 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52495ba8-7799-4831-ac11-34d4cbd5ee3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:09.709092 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52495ba8-7799-4831-ac11-34d4cbd5ee3c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:09.711067 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 105/201] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/role_assignments?user.id=ae2439475b714dd1ba40dbe86303d708&scope.project.id=359a065a7d2c49fea870a82587a5d51e&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:09.712207 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f9cb3d-d3c9-4c87-8f1b-4d891126d5c6 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.739567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 97/202] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:09.746067 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52495ba8-7799-4831-ac11-34d4cbd5ee3c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.831526 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:09.839956 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f9cb3d-d3c9-4c87-8f1b-4d891126d5c6 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.887793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ecf44-9922-4d55-8c52-66eb4d4b1043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.888634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ecf44-9922-4d55-8c52-66eb4d4b1043 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.888921 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ecf44-9922-4d55-8c52-66eb4d4b1043 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.889200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ecf44-9922-4d55-8c52-66eb4d4b1043 None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.890527 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ecf44-9922-4d55-8c52-66eb4d4b1043 None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:09.891982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ecf44-9922-4d55-8c52-66eb4d4b1043 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:09.893730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 106/203] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:09.901045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ecf44-9922-4d55-8c52-66eb4d4b1043 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.947469 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:09.953429 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52495ba8-7799-4831-ac11-34d4cbd5ee3c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:10.028127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426ccfe-911b-4fa6-ab75-57d0f5db0d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:10.029029 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426ccfe-911b-4fa6-ab75-57d0f5db0d4a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:10.029308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426ccfe-911b-4fa6-ab75-57d0f5db0d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:10.029586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426ccfe-911b-4fa6-ab75-57d0f5db0d4a None admin] PATH_INFO: `/v3/users/717605caf5e34451b2fac90d3a2c249a` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:10.033896 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426ccfe-911b-4fa6-ab75-57d0f5db0d4a None admin] RBAC: Authorizing `identity:get_user(user_id=717605caf5e34451b2fac90d3a2c249a)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:10.035358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426ccfe-911b-4fa6-ab75-57d0f5db0d4a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:10.045307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 98/204] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/users/717605caf5e34451b2fac90d3a2c249a => generated 289 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:10.046355 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426ccfe-911b-4fa6-ab75-57d0f5db0d4a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:10.097104 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ecf44-9922-4d55-8c52-66eb4d4b1043 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:10.165531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369d8fb-dc73-4cf8-af91-6b937467aa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:10.166108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369d8fb-dc73-4cf8-af91-6b937467aa2e None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:10.166395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369d8fb-dc73-4cf8-af91-6b937467aa2e None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:10.166671 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369d8fb-dc73-4cf8-af91-6b937467aa2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:10.167517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369d8fb-dc73-4cf8-af91-6b937467aa2e None admin] RBAC: Adding query filter params (role.id=abe200f23fc84ba397dd8c67874f0f2a, scope.project.id=359a065a7d2c49fea870a82587a5d51e, user.id=ae2439475b714dd1ba40dbe86303d708) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:10.167831 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369d8fb-dc73-4cf8-af91-6b937467aa2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:10.169296 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369d8fb-dc73-4cf8-af91-6b937467aa2e None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:10.186259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 107/205] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/role_assignments?user.id=ae2439475b714dd1ba40dbe86303d708&scope.project.id=359a065a7d2c49fea870a82587a5d51e&role.id=abe200f23fc84ba397dd8c67874f0f2a => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:10.242277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426ccfe-911b-4fa6-ab75-57d0f5db0d4a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:10.297065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acdee2b-b6bf-47c1-967e-3483d7939b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:10.297738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdee2b-b6bf-47c1-967e-3483d7939b9d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:10.298026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdee2b-b6bf-47c1-967e-3483d7939b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:10.298652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdee2b-b6bf-47c1-967e-3483d7939b9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:10.346272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:10.406056 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acdee2b-b6bf-47c1-967e-3483d7939b9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:10.407586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acdee2b-b6bf-47c1-967e-3483d7939b9d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:10.531369 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 99/206] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:10.537471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369d8fb-dc73-4cf8-af91-6b937467aa2e None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:10.574137 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:10.663199 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369d8fb-dc73-4cf8-af91-6b937467aa2e None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:10.709969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d52ef-39c3-435b-b4cf-f1615ccda31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:10.711421 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d52ef-39c3-435b-b4cf-f1615ccda31a None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:10.711703 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d52ef-39c3-435b-b4cf-f1615ccda31a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:10.711975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d52ef-39c3-435b-b4cf-f1615ccda31a None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:10.713222 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d52ef-39c3-435b-b4cf-f1615ccda31a None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:10.714678 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d52ef-39c3-435b-b4cf-f1615ccda31a None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:10.716699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 108/207] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:10 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:10.722201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acdee2b-b6bf-47c1-967e-3483d7939b9d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:10.804025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:10.857529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acdee2b-b6bf-47c1-967e-3483d7939b9d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:10.910378 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7128693-981a-4ffe-a2a0-98e11b24f784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:10.911043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7128693-981a-4ffe-a2a0-98e11b24f784 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:10.911322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7128693-981a-4ffe-a2a0-98e11b24f784 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:10.911594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7128693-981a-4ffe-a2a0-98e11b24f784 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:10.912473 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7128693-981a-4ffe-a2a0-98e11b24f784 None admin] RBAC: Adding query filter params (role.id=ba4641dd50c64f80a618b95aaaa321cb, scope.project.id=711e8b07480d478a80cbefc6fe30467f, user.id=717605caf5e34451b2fac90d3a2c249a) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:10.912785 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7128693-981a-4ffe-a2a0-98e11b24f784 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:10.914797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7128693-981a-4ffe-a2a0-98e11b24f784 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:10.925289 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 100/208] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:10 2022] GET /identity/v3/role_assignments?user.id=717605caf5e34451b2fac90d3a2c249a&scope.project.id=711e8b07480d478a80cbefc6fe30467f&role.id=ba4641dd50c64f80a618b95aaaa321cb => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:11.493711 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c2c39-4475-4fe0-b897-d64d983b5349 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:11.493711 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c2c39-4475-4fe0-b897-d64d983b5349 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:11.493711 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c2c39-4475-4fe0-b897-d64d983b5349 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:11.494927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 109/209] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:11.501287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ca16d-e631-429a-b622-e7527135bf49 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:11.501917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ca16d-e631-429a-b622-e7527135bf49 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:11.502401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ca16d-e631-429a-b622-e7527135bf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:11.523958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-858ca16d-e631-429a-b622-e7527135bf49 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:11.525556 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-858ca16d-e631-429a-b622-e7527135bf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:11.602946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d191a13-6500-4a4a-80e8-c10f74e752e5 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:11.603133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d191a13-6500-4a4a-80e8-c10f74e752e5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:11.603411 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d191a13-6500-4a4a-80e8-c10f74e752e5 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:11.605070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 110/210] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:11.610408 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd530cb-d17e-4d01-874b-77d7735fee77 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:11.610693 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd530cb-d17e-4d01-874b-77d7735fee77 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:11.610968 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd530cb-d17e-4d01-874b-77d7735fee77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:11.618206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 101/211] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:11.632769 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4fd530cb-d17e-4d01-874b-77d7735fee77 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:11.634231 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4fd530cb-d17e-4d01-874b-77d7735fee77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:11.641117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ca16d-e631-429a-b622-e7527135bf49 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:11.721866 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:11.726667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 111/212] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:11.751532 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd530cb-d17e-4d01-874b-77d7735fee77 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:11.766778 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ca16d-e631-429a-b622-e7527135bf49 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:11.779675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:11.805635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd7e21-8a37-4b51-8e4a-93fff335c76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:11.806438 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd7e21-8a37-4b51-8e4a-93fff335c76e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:11.806727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd7e21-8a37-4b51-8e4a-93fff335c76e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:11.807027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd7e21-8a37-4b51-8e4a-93fff335c76e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:11.818509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd7e21-8a37-4b51-8e4a-93fff335c76e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:11.826485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-abdd7e21-8a37-4b51-8e4a-93fff335c76e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:11.826485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:11.826485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:11.826485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:11.826485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:11.826485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:11.826485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:11.826485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:11.826485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:11.826485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:11.826485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:11.826485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:11.826485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:11.826485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:11.826485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:11.826485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:11.827381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:11.827381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:11.827381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:11.827381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:11.827381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:11.827381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:11.827381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:11.827381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:11.827381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:11.827381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:11.827381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:11.827381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:11.827381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:11.827381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:11.827381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:11.827381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:11.828225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:11.828225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:11.828225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:11.828225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:11.828225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:11.828225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:11.828225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:11.828225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:11.828225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:11.828225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:11.828225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:11.828225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:11.828225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 102/213] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:11.833466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd7e21-8a37-4b51-8e4a-93fff335c76e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:11.880257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd530cb-d17e-4d01-874b-77d7735fee77 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:11.918118 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79898ee3-0d20-4ff2-a05a-cd1b327bfea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:11.919255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79898ee3-0d20-4ff2-a05a-cd1b327bfea6 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:11.919718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79898ee3-0d20-4ff2-a05a-cd1b327bfea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:11.920164 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79898ee3-0d20-4ff2-a05a-cd1b327bfea6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:11.925073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79898ee3-0d20-4ff2-a05a-cd1b327bfea6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:11.927710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79898ee3-0d20-4ff2-a05a-cd1b327bfea6 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:11.934464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 112/214] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:11.935501 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79898ee3-0d20-4ff2-a05a-cd1b327bfea6 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:11.953719 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd7e21-8a37-4b51-8e4a-93fff335c76e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:11.978104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:11.989589 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764b436-8643-4773-b30a-d235108f58e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:11.990259 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764b436-8643-4773-b30a-d235108f58e6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:11.990544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764b436-8643-4773-b30a-d235108f58e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:11.990823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764b436-8643-4773-b30a-d235108f58e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:11.991617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764b436-8643-4773-b30a-d235108f58e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:11.991954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764b436-8643-4773-b30a-d235108f58e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:11.993692 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764b436-8643-4773-b30a-d235108f58e6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:12.002022 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 103/215] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:12.007733 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764b436-8643-4773-b30a-d235108f58e6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:12.035614 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:12.046783 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79898ee3-0d20-4ff2-a05a-cd1b327bfea6 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:12.092159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729ce4c-56fe-49b3-8120-449f5e6ef5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:12.092882 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729ce4c-56fe-49b3-8120-449f5e6ef5bd None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:12.093214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729ce4c-56fe-49b3-8120-449f5e6ef5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:12.093483 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729ce4c-56fe-49b3-8120-449f5e6ef5bd None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:12.094257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729ce4c-56fe-49b3-8120-449f5e6ef5bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:12.095809 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729ce4c-56fe-49b3-8120-449f5e6ef5bd None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:12.119917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 113/216] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:10:11 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:12.138161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764b436-8643-4773-b30a-d235108f58e6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:12.179342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078f590-b06d-4c09-a02e-b3c36aa28dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:12.180251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078f590-b06d-4c09-a02e-b3c36aa28dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:12.180559 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078f590-b06d-4c09-a02e-b3c36aa28dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:12.180837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078f590-b06d-4c09-a02e-b3c36aa28dd8 None admin] PATH_INFO: `/v3/users/b62462cb1ace4a2f805075406ec7b68d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:12.199041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078f590-b06d-4c09-a02e-b3c36aa28dd8 None admin] RBAC: Authorizing `identity:get_user(user_id=b62462cb1ace4a2f805075406ec7b68d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:12.200786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078f590-b06d-4c09-a02e-b3c36aa28dd8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:12.202793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 104/217] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:12 2022] GET /identity/v3/users/b62462cb1ace4a2f805075406ec7b68d => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:12.208676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729ce4c-56fe-49b3-8120-449f5e6ef5bd None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:12.225548 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6df32c-1287-495f-b2ca-9e5c608d347c None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:12.225897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6df32c-1287-495f-b2ca-9e5c608d347c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:12.226168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6df32c-1287-495f-b2ca-9e5c608d347c None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:12.227735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 105/218] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:12.233006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b749096-c0e8-406f-aadd-51c794fc908a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:12.233320 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b749096-c0e8-406f-aadd-51c794fc908a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:12.233570 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b749096-c0e8-406f-aadd-51c794fc908a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:12.254593 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7b749096-c0e8-406f-aadd-51c794fc908a None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:12.256572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7b749096-c0e8-406f-aadd-51c794fc908a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:12.302237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:12.339811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729ce4c-56fe-49b3-8120-449f5e6ef5bd None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:12.352166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 106/219] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:12.375294 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b749096-c0e8-406f-aadd-51c794fc908a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:12.385351 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a650031-ec20-455e-b79a-33e5a4b25885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:12.385538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a650031-ec20-455e-b79a-33e5a4b25885 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:12.386248 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a650031-ec20-455e-b79a-33e5a4b25885 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:12.386248 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a650031-ec20-455e-b79a-33e5a4b25885 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:12.406359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:12.482546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a650031-ec20-455e-b79a-33e5a4b25885 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:12.484724 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a650031-ec20-455e-b79a-33e5a4b25885 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:12.513140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b749096-c0e8-406f-aadd-51c794fc908a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:12.559497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90350675-88e0-46d9-9648-eedab9afdc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:12.560338 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90350675-88e0-46d9-9648-eedab9afdc7b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:12.560628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90350675-88e0-46d9-9648-eedab9afdc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:12.560905 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90350675-88e0-46d9-9648-eedab9afdc7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:12.573253 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90350675-88e0-46d9-9648-eedab9afdc7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:12.581440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-90350675-88e0-46d9-9648-eedab9afdc7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:12.581440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:12.581440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:12.581440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:12.581440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:12.581440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:12.581440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:12.581440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:12.581440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:12.581440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:12.581440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:12.581440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:12.581440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:12.581440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:12.581440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:12.581440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:12.582932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:12.582932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:12.582932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:12.582932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:12.582932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:12.582932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:12.582932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:12.582932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:12.582932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:12.582932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:12.582932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:12.582932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:12.582932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:12.582932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:12.582932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:12.582932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:12.584122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:12.584122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:12.584122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:12.584122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:12.584122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:12.584122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:12.584122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:12.584122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:12.584122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:12.584122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:12.584122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:12.584122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:12.584122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 107/220] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:12.593310 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90350675-88e0-46d9-9648-eedab9afdc7b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:12.638340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 114/221] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:12.641021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:12.644146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a650031-ec20-455e-b79a-33e5a4b25885 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:12.705449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:12.745871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90350675-88e0-46d9-9648-eedab9afdc7b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:12.763223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a650031-ec20-455e-b79a-33e5a4b25885 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:12.791880 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6af28d-c661-464e-ac4d-5cb5f1c4c210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:12.792595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6af28d-c661-464e-ac4d-5cb5f1c4c210 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:12.792881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6af28d-c661-464e-ac4d-5cb5f1c4c210 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:12.793296 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6af28d-c661-464e-ac4d-5cb5f1c4c210 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:12.794197 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6af28d-c661-464e-ac4d-5cb5f1c4c210 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:12.794507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6af28d-c661-464e-ac4d-5cb5f1c4c210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:12.796836 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6af28d-c661-464e-ac4d-5cb5f1c4c210 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:12.805005 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8c8cb-ba72-4d0e-bea8-1eabc1d21da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:12.806154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8c8cb-ba72-4d0e-bea8-1eabc1d21da5 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:12.806632 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8c8cb-ba72-4d0e-bea8-1eabc1d21da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:12.807118 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8c8cb-ba72-4d0e-bea8-1eabc1d21da5 None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:12.812352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 108/222] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:12.812352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1763095-3743-4639-9a3f-121341f6771d None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:12.812352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1763095-3743-4639-9a3f-121341f6771d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:12.812352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1763095-3743-4639-9a3f-121341f6771d None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:12.813763 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8c8cb-ba72-4d0e-bea8-1eabc1d21da5 None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:12.815494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8c8cb-ba72-4d0e-bea8-1eabc1d21da5 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:12.815600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 109/223] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:12.816802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1763095-3743-4639-9a3f-121341f6771d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:12.818616 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 115/224] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:12 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:12.820187 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1c42d-34b3-4ea0-be71-829765a64e96 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:12.820789 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1c42d-34b3-4ea0-be71-829765a64e96 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:12.821258 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1c42d-34b3-4ea0-be71-829765a64e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:12.839643 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-bdb1c42d-34b3-4ea0-be71-829765a64e96 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:12.841192 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-bdb1c42d-34b3-4ea0-be71-829765a64e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:12.921002 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:12.933238 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1763095-3743-4639-9a3f-121341f6771d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:12.945035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 116/225] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:12.945035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1c42d-34b3-4ea0-be71-829765a64e96 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:12.988557 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:12.993067 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae26f6d-5e47-4e38-8072-0ffa8e7c8517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:12.994192 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae26f6d-5e47-4e38-8072-0ffa8e7c8517 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:12.994667 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae26f6d-5e47-4e38-8072-0ffa8e7c8517 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:12.995133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae26f6d-5e47-4e38-8072-0ffa8e7c8517 None admin] PATH_INFO: `/v3/users/ae2439475b714dd1ba40dbe86303d708` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:12.996541 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae26f6d-5e47-4e38-8072-0ffa8e7c8517 None admin] RBAC: Authorizing `identity:get_user(user_id=ae2439475b714dd1ba40dbe86303d708)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:12.998192 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae26f6d-5e47-4e38-8072-0ffa8e7c8517 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:13.000067 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 110/226] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:12 2022] GET /identity/v3/users/ae2439475b714dd1ba40dbe86303d708 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:13.005413 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae26f6d-5e47-4e38-8072-0ffa8e7c8517 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:13.049724 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1c42d-34b3-4ea0-be71-829765a64e96 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:13.088433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f0fcf-8373-4f28-9c64-c48b11fabe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:13.089092 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f0fcf-8373-4f28-9c64-c48b11fabe04 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:13.089374 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f0fcf-8373-4f28-9c64-c48b11fabe04 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:13.089649 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f0fcf-8373-4f28-9c64-c48b11fabe04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:13.090775 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f0fcf-8373-4f28-9c64-c48b11fabe04 None admin] RBAC: Adding query filter params (role.id=18f0dc41b78c4b52b6912225928ea388, scope.project.id=359a065a7d2c49fea870a82587a5d51e, user.id=b62462cb1ace4a2f805075406ec7b68d) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:13.090883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f0fcf-8373-4f28-9c64-c48b11fabe04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:13.092748 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f0fcf-8373-4f28-9c64-c48b11fabe04 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:13.099023 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 117/227] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:12 2022] GET /identity/v3/role_assignments?user.id=b62462cb1ace4a2f805075406ec7b68d&scope.project.id=359a065a7d2c49fea870a82587a5d51e&role.id=18f0dc41b78c4b52b6912225928ea388 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:13.100253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f0fcf-8373-4f28-9c64-c48b11fabe04 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:13.119707 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae26f6d-5e47-4e38-8072-0ffa8e7c8517 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:13.169926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5afa66-2c8f-4edf-8c73-2aed8beffd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:13.171713 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5afa66-2c8f-4edf-8c73-2aed8beffd6e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:13.172455 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5afa66-2c8f-4edf-8c73-2aed8beffd6e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:13.172942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5afa66-2c8f-4edf-8c73-2aed8beffd6e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:13.187126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5afa66-2c8f-4edf-8c73-2aed8beffd6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:13.198711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-5e5afa66-2c8f-4edf-8c73-2aed8beffd6e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:13.198711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:13.198711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:13.198711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:13.198711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:13.198711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:13.198711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:13.198711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:13.198711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:13.198711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:13.198711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:13.198711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:13.198711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:13.198711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:13.198711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:13.198711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:13.199668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:13.199668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:13.199668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:13.199668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:13.199668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:13.199668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:13.199668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:13.199668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:13.199668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:13.199668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:13.199668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:13.199668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:13.199668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:13.199668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:13.199668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:13.199668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:13.200489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:13.200489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:13.200489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:13.200489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:13.200489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:13.200489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:13.200489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:13.200489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:13.200489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:13.200489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:13.200489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:13.200489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:13.203071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 111/228] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:13.209179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5afa66-2c8f-4edf-8c73-2aed8beffd6e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:13.229652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f0fcf-8373-4f28-9c64-c48b11fabe04 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:13.235051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:13.242836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:13.273488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b88e7a-e156-44a4-a41e-fc915f368fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:13.274711 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b88e7a-e156-44a4-a41e-fc915f368fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:13.275172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b88e7a-e156-44a4-a41e-fc915f368fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:13.275699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b88e7a-e156-44a4-a41e-fc915f368fa0 None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:13.277131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b88e7a-e156-44a4-a41e-fc915f368fa0 None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:13.278737 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b88e7a-e156-44a4-a41e-fc915f368fa0 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:13.280843 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 118/229] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:13 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:13.288275 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b88e7a-e156-44a4-a41e-fc915f368fa0 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:13.353921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5afa66-2c8f-4edf-8c73-2aed8beffd6e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:13.405540 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231cf97-de7e-40ae-9050-7b5377608f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:13.406366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231cf97-de7e-40ae-9050-7b5377608f65 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:13.406646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231cf97-de7e-40ae-9050-7b5377608f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:13.406920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231cf97-de7e-40ae-9050-7b5377608f65 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:13.407681 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231cf97-de7e-40ae-9050-7b5377608f65 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:13.407985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231cf97-de7e-40ae-9050-7b5377608f65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:13.412167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231cf97-de7e-40ae-9050-7b5377608f65 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:13.425042 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 112/230] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:13.435337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231cf97-de7e-40ae-9050-7b5377608f65 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:13.537759 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:13.539112 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b88e7a-e156-44a4-a41e-fc915f368fa0 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:13.560878 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231cf97-de7e-40ae-9050-7b5377608f65 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:13.609908 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eeccbb-f5cd-49d9-bf6c-594b97c73e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:13.611329 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eeccbb-f5cd-49d9-bf6c-594b97c73e6d None admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:13.611766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eeccbb-f5cd-49d9-bf6c-594b97c73e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:13.612343 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eeccbb-f5cd-49d9-bf6c-594b97c73e6d None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e/users/ae2439475b714dd1ba40dbe86303d708/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:13.614881 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eeccbb-f5cd-49d9-bf6c-594b97c73e6d None admin] RBAC: Authorizing `identity:create_grant(project_id=359a065a7d2c49fea870a82587a5d51e, user_id=ae2439475b714dd1ba40dbe86303d708, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:13.616694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eeccbb-f5cd-49d9-bf6c-594b97c73e6d None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:13.631127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e77f6-f3da-4e32-af61-a39d75212636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:13.632858 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e77f6-f3da-4e32-af61-a39d75212636 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:13.633176 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e77f6-f3da-4e32-af61-a39d75212636 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:13.633544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e77f6-f3da-4e32-af61-a39d75212636 None admin] PATH_INFO: `/v3/users/ae2439475b714dd1ba40dbe86303d708` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:13.634945 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e77f6-f3da-4e32-af61-a39d75212636 None admin] RBAC: Authorizing `identity:get_user(user_id=ae2439475b714dd1ba40dbe86303d708)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:13.642603 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e77f6-f3da-4e32-af61-a39d75212636 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:13.642603 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 113/231] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:13 2022] GET /identity/v3/users/ae2439475b714dd1ba40dbe86303d708 => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:13.643841 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e77f6-f3da-4e32-af61-a39d75212636 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:13.646185 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 119/232] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:13 2022] PUT /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e/users/ae2439475b714dd1ba40dbe86303d708/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:10:13.761631 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e77f6-f3da-4e32-af61-a39d75212636 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:13.798283 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f821f-d6f8-4ec8-8e1e-1b2d1fd12349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:13.799075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f821f-d6f8-4ec8-8e1e-1b2d1fd12349 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:13.799361 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f821f-d6f8-4ec8-8e1e-1b2d1fd12349 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:13.799641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f821f-d6f8-4ec8-8e1e-1b2d1fd12349 None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:13.800960 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f821f-d6f8-4ec8-8e1e-1b2d1fd12349 None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:13.802482 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f821f-d6f8-4ec8-8e1e-1b2d1fd12349 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:13.805230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 114/233] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:13 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:13.813465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eeccbb-f5cd-49d9-bf6c-594b97c73e6d None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:13.883760 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:13.961302 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eeccbb-f5cd-49d9-bf6c-594b97c73e6d None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:14.001064 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057e322-9bc8-4c5d-95d6-276ba4b916e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:14.001970 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057e322-9bc8-4c5d-95d6-276ba4b916e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:14.002252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057e322-9bc8-4c5d-95d6-276ba4b916e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:14.002529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057e322-9bc8-4c5d-95d6-276ba4b916e3 None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e/users/ae2439475b714dd1ba40dbe86303d708/roles/abe200f23fc84ba397dd8c67874f0f2a` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:14.010669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057e322-9bc8-4c5d-95d6-276ba4b916e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=359a065a7d2c49fea870a82587a5d51e, user_id=ae2439475b714dd1ba40dbe86303d708, role_id=abe200f23fc84ba397dd8c67874f0f2a)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:14.012269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057e322-9bc8-4c5d-95d6-276ba4b916e3 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:14.045958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 120/234] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:13 2022] PUT /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e/users/ae2439475b714dd1ba40dbe86303d708/roles/abe200f23fc84ba397dd8c67874f0f2a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:14.649289 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfbb96-8a06-4a3e-a9ec-24dc8c9a4185 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:14.649623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfbb96-8a06-4a3e-a9ec-24dc8c9a4185 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:14.649796 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfbb96-8a06-4a3e-a9ec-24dc8c9a4185 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:14.651407 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 115/235] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:14.660819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8eaf1c-0693-43b3-bd30-9f9b8da492ad None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:14.661301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8eaf1c-0693-43b3-bd30-9f9b8da492ad None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:14.661376 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8eaf1c-0693-43b3-bd30-9f9b8da492ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:14.681115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-5f8eaf1c-0693-43b3-bd30-9f9b8da492ad None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:14.682307 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5f8eaf1c-0693-43b3-bd30-9f9b8da492ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:14.764416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 121/236] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:14.787679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfbb96-8a06-4a3e-a9ec-24dc8c9a4185 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:14.808108 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:14.894767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfbb96-8a06-4a3e-a9ec-24dc8c9a4185 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:14.929291 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d213c3b-e5b6-43ed-9627-499feb953ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:14.930011 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d213c3b-e5b6-43ed-9627-499feb953ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:14.930292 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d213c3b-e5b6-43ed-9627-499feb953ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:14.930568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d213c3b-e5b6-43ed-9627-499feb953ca2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:14.941388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d213c3b-e5b6-43ed-9627-499feb953ca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:14.949789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-3d213c3b-e5b6-43ed-9627-499feb953ca2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:14.949789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:14.949789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:14.949789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:14.949789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:14.949789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:14.949789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:14.949789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:14.949789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:14.949789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:14.949789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:14.949789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:14.949789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:14.949789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:14.949789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:14.949789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:14.950701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:14.950701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:14.950701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:14.950701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:14.950701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:14.950701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:14.950701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:14.950701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:14.950701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:14.950701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:14.950701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:14.950701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:14.950701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:14.950701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:14.950701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:14.950701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:14.951467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:14.951467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:14.951467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:14.951467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:14.951467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:14.951467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:14.951467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:14.951467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:14.951467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:14.951467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:14.951467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:14.951467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:14.951467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 116/237] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:14.967352 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8eaf1c-0693-43b3-bd30-9f9b8da492ad None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:14.995156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:15.065605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d544f-7b05-4bf9-b0e2-845318b85ac8 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.066062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d544f-7b05-4bf9-b0e2-845318b85ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.066494 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d544f-7b05-4bf9-b0e2-845318b85ac8 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.068168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 117/238] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:15.073533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bd476-399e-4a89-97b6-201319238dc0 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.073826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bd476-399e-4a89-97b6-201319238dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.074104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bd476-399e-4a89-97b6-201319238dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.092421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5d5bd476-399e-4a89-97b6-201319238dc0 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:15.093626 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5d5bd476-399e-4a89-97b6-201319238dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:15.105562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8eaf1c-0693-43b3-bd30-9f9b8da492ad None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:15.165504 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 118/239] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:15.191144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a17c0-0333-4648-8e5e-94201c1d11f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:15.191635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bd476-399e-4a89-97b6-201319238dc0 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:15.192409 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a17c0-0333-4648-8e5e-94201c1d11f7 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.192908 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a17c0-0333-4648-8e5e-94201c1d11f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.193372 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a17c0-0333-4648-8e5e-94201c1d11f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.194944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a17c0-0333-4648-8e5e-94201c1d11f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:15.195508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a17c0-0333-4648-8e5e-94201c1d11f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:15.197118 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a17c0-0333-4648-8e5e-94201c1d11f7 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:15.202894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:15.212025 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 122/240] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:15.219799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a17c0-0333-4648-8e5e-94201c1d11f7 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:15.304533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bd476-399e-4a89-97b6-201319238dc0 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:15.351546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:15.356329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843098ca-4662-4f18-be1c-403348f12488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:15.357384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843098ca-4662-4f18-be1c-403348f12488 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.357865 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843098ca-4662-4f18-be1c-403348f12488 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.358334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843098ca-4662-4f18-be1c-403348f12488 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.360079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843098ca-4662-4f18-be1c-403348f12488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:15.361765 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843098ca-4662-4f18-be1c-403348f12488 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:15.363614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 119/241] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:15.369768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843098ca-4662-4f18-be1c-403348f12488 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:15.373153 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a17c0-0333-4648-8e5e-94201c1d11f7 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:15.465109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8278827-8575-4302-ae9c-b380a9bab2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:15.465930 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8278827-8575-4302-ae9c-b380a9bab2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.466219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8278827-8575-4302-ae9c-b380a9bab2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.466676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8278827-8575-4302-ae9c-b380a9bab2c5 None admin] PATH_INFO: `/v3/users/a176881b0e804295849981bb76bd5391` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.475570 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843098ca-4662-4f18-be1c-403348f12488 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:15.483788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:15.502972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8278827-8575-4302-ae9c-b380a9bab2c5 None admin] RBAC: Authorizing `identity:get_user(user_id=a176881b0e804295849981bb76bd5391)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:15.504647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8278827-8575-4302-ae9c-b380a9bab2c5 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:15.506546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 123/242] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:15 2022] GET /identity/v3/users/a176881b0e804295849981bb76bd5391 => generated 297 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:15.518746 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08354c1-929c-4b96-8b1a-395f9c262015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:15.519821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08354c1-929c-4b96-8b1a-395f9c262015 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.520394 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08354c1-929c-4b96-8b1a-395f9c262015 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.520871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08354c1-929c-4b96-8b1a-395f9c262015 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.521327 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8278827-8575-4302-ae9c-b380a9bab2c5 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:15.522124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08354c1-929c-4b96-8b1a-395f9c262015 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:15.523889 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08354c1-929c-4b96-8b1a-395f9c262015 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:15.557837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 120/243] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:10:15 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:10:15.637985 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8278827-8575-4302-ae9c-b380a9bab2c5 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:15.688121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9ec09-8ddd-459d-9dbd-63c4ec5bdf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:15.688961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9ec09-8ddd-459d-9dbd-63c4ec5bdf86 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.689152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9ec09-8ddd-459d-9dbd-63c4ec5bdf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.689388 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9ec09-8ddd-459d-9dbd-63c4ec5bdf86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.729500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:15.741300 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e844e0-e2bc-4080-a1a3-d4741f816abd None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.742402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e844e0-e2bc-4080-a1a3-d4741f816abd None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.742877 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e844e0-e2bc-4080-a1a3-d4741f816abd None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.744838 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 121/244] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:15.750721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa517462-0803-480e-bf51-07769d878aff None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.751375 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa517462-0803-480e-bf51-07769d878aff None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.751853 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa517462-0803-480e-bf51-07769d878aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.775760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-aa517462-0803-480e-bf51-07769d878aff None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:15.781694 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-aa517462-0803-480e-bf51-07769d878aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:15.796844 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9ec09-8ddd-459d-9dbd-63c4ec5bdf86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:15.798618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9ec09-8ddd-459d-9dbd-63c4ec5bdf86 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:15.877845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 122/245] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:15.900731 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa517462-0803-480e-bf51-07769d878aff None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:15.921594 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 124/246] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:10:15.927135 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9ec09-8ddd-459d-9dbd-63c4ec5bdf86 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:15.971721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:15.983811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:16.023790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa517462-0803-480e-bf51-07769d878aff None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:16.049833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9ec09-8ddd-459d-9dbd-63c4ec5bdf86 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:16.066679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ac716-4a5f-4cbd-a62a-b6becd92809c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:16.067862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ac716-4a5f-4cbd-a62a-b6becd92809c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:16.068312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ac716-4a5f-4cbd-a62a-b6becd92809c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:16.068729 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ac716-4a5f-4cbd-a62a-b6becd92809c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:16.082225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ac716-4a5f-4cbd-a62a-b6becd92809c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:16.091609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27062a-7bff-4f7a-98a3-c334249664c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:16.092653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27062a-7bff-4f7a-98a3-c334249664c4 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:16.093042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27062a-7bff-4f7a-98a3-c334249664c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:16.093421 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27062a-7bff-4f7a-98a3-c334249664c4 None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:16.093735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-777ac716-4a5f-4cbd-a62a-b6becd92809c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:16.093735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:16.093735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:16.093735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:16.093735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:16.093735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:16.093735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:16.093735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:16.093735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:16.093735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:16.093735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:16.093735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:16.093735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:16.093735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:16.093735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:16.093735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:16.094797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:16.094797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:16.094797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:16.094797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:16.094797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:16.094797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:16.094797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:16.094797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:16.094797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:16.094797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:16.094797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:16.094797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:16.094797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:16.094797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:16.094797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:16.094797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:16.095767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:16.095767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:16.095767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:16.095767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:16.095767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:16.095767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:16.095767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:16.095767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:16.095767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:16.095767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:16.095767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:16.095767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:16.097276 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27062a-7bff-4f7a-98a3-c334249664c4 None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:16.098420 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 123/247] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:16.101463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27062a-7bff-4f7a-98a3-c334249664c4 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:16.101463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 125/248] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:15 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:16.105349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ac716-4a5f-4cbd-a62a-b6becd92809c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:16.107243 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27062a-7bff-4f7a-98a3-c334249664c4 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:16.238423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27062a-7bff-4f7a-98a3-c334249664c4 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:16.239881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ac716-4a5f-4cbd-a62a-b6becd92809c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:16.258360 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:16.268584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:16.284108 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf521c6-fc21-4623-9dd8-95a649651454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:16.284846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1918456-144a-42e5-9f94-a02721b57022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:16.285211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf521c6-fc21-4623-9dd8-95a649651454 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:16.285676 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf521c6-fc21-4623-9dd8-95a649651454 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:16.285883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1918456-144a-42e5-9f94-a02721b57022 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:16.286183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1918456-144a-42e5-9f94-a02721b57022 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:16.286463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf521c6-fc21-4623-9dd8-95a649651454 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:16.286643 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1918456-144a-42e5-9f94-a02721b57022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:16.287504 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf521c6-fc21-4623-9dd8-95a649651454 None admin] RBAC: Adding query filter params (name=member) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:16.287659 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1918456-144a-42e5-9f94-a02721b57022 None admin] RBAC: Adding query filter params (role.id=ba4641dd50c64f80a618b95aaaa321cb, scope.project.id=359a065a7d2c49fea870a82587a5d51e, user.id=a176881b0e804295849981bb76bd5391) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:16.288661 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1918456-144a-42e5-9f94-a02721b57022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:16.288763 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf521c6-fc21-4623-9dd8-95a649651454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:16.290355 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf521c6-fc21-4623-9dd8-95a649651454 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:16.291077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1918456-144a-42e5-9f94-a02721b57022 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:16.298707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 126/249] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:16 2022] GET /identity/v3/role_assignments?user.id=a176881b0e804295849981bb76bd5391&scope.project.id=359a065a7d2c49fea870a82587a5d51e&role.id=ba4641dd50c64f80a618b95aaaa321cb => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:16.301078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 124/250] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:16.307066 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1918456-144a-42e5-9f94-a02721b57022 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:16.339105 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a58a4-69b2-4078-ab78-ae732a6e0ee3 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:16.339626 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a58a4-69b2-4078-ab78-ae732a6e0ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:16.340048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a58a4-69b2-4078-ab78-ae732a6e0ee3 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:16.348610 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 125/251] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:16 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:16.355316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9ba35-057a-4b3e-be22-1d2178f7c82e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:16.355742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9ba35-057a-4b3e-be22-1d2178f7c82e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:16.356012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9ba35-057a-4b3e-be22-1d2178f7c82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:16.386566 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-80a9ba35-057a-4b3e-be22-1d2178f7c82e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:16.388026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-80a9ba35-057a-4b3e-be22-1d2178f7c82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:16.483439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1918456-144a-42e5-9f94-a02721b57022 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:16.516726 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 126/252] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:16.535890 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dd325-a42f-4585-a5b8-b3a1d1755035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:16.536830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dd325-a42f-4585-a5b8-b3a1d1755035 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:16.537113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dd325-a42f-4585-a5b8-b3a1d1755035 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:16.537388 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dd325-a42f-4585-a5b8-b3a1d1755035 None admin] PATH_INFO: `/v3/users/b62462cb1ace4a2f805075406ec7b68d` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:16.539399 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dd325-a42f-4585-a5b8-b3a1d1755035 None admin] RBAC: Authorizing `identity:get_user(user_id=b62462cb1ace4a2f805075406ec7b68d)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:16.540920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dd325-a42f-4585-a5b8-b3a1d1755035 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:16.544447 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 127/253] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:16 2022] GET /identity/v3/users/b62462cb1ace4a2f805075406ec7b68d => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:16.548456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9ba35-057a-4b3e-be22-1d2178f7c82e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:16.554669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dd325-a42f-4585-a5b8-b3a1d1755035 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:16.610588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:16.668382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9ba35-057a-4b3e-be22-1d2178f7c82e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:16.670095 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dd325-a42f-4585-a5b8-b3a1d1755035 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:16.683579 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:16.718681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a0747-bdf8-433c-a70b-f3d016a85e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:16.719501 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a0747-bdf8-433c-a70b-f3d016a85e3f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:16.719791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a0747-bdf8-433c-a70b-f3d016a85e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:16.720071 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a0747-bdf8-433c-a70b-f3d016a85e3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:16.729251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0c93e-8e1d-4b8a-b6ef-2bb7ab91b115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:16.730131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0c93e-8e1d-4b8a-b6ef-2bb7ab91b115 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:16.730417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0c93e-8e1d-4b8a-b6ef-2bb7ab91b115 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:16.730690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0c93e-8e1d-4b8a-b6ef-2bb7ab91b115 None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:16.732194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0c93e-8e1d-4b8a-b6ef-2bb7ab91b115 None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:16.733736 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0c93e-8e1d-4b8a-b6ef-2bb7ab91b115 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:16.736956 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 127/254] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:16 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:16.742731 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0c93e-8e1d-4b8a-b6ef-2bb7ab91b115 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:16.744407 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a0747-bdf8-433c-a70b-f3d016a85e3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:16.754498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-ed6a0747-bdf8-433c-a70b-f3d016a85e3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:16.754498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:16.754498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:16.754498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:16.754498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:16.754498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:16.754498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:16.754498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:16.754498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:16.754498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:16.754498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:16.754498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:16.754498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:16.754498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:16.754498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:16.754498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:16.755568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:16.755568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:16.755568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:16.755568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:16.755568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:16.755568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:16.755568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:16.755568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:16.755568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:16.755568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:16.755568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:16.755568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:16.755568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:16.755568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:16.755568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:16.755568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:16.756511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:16.756511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:16.756511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:16.756511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:16.756511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:16.756511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:16.756511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:16.756511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:16.756511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:16.756511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:16.756511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:16.756511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:16.763800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 128/255] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:16.771284 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78bed5-454b-4ea4-810b-b574c0ac72f7 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:16.771502 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78bed5-454b-4ea4-810b-b574c0ac72f7 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:16.771790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78bed5-454b-4ea4-810b-b574c0ac72f7 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:16.773891 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 129/256] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:16.777729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78bed5-454b-4ea4-810b-b574c0ac72f7 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:16.889197 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0c93e-8e1d-4b8a-b6ef-2bb7ab91b115 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:16.916379 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78bed5-454b-4ea4-810b-b574c0ac72f7 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:16.925907 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e27619-2dd8-440a-9587-d98e6371a01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:16.926451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:16.927047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e27619-2dd8-440a-9587-d98e6371a01c None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:16.927429 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e27619-2dd8-440a-9587-d98e6371a01c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:16.927802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e27619-2dd8-440a-9587-d98e6371a01c None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e/users/b62462cb1ace4a2f805075406ec7b68d/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:16.930006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e27619-2dd8-440a-9587-d98e6371a01c None admin] RBAC: Authorizing `identity:create_grant(project_id=359a065a7d2c49fea870a82587a5d51e, user_id=b62462cb1ace4a2f805075406ec7b68d, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:16.931579 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e27619-2dd8-440a-9587-d98e6371a01c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:16.957844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 128/257] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:16 2022] PUT /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e/users/b62462cb1ace4a2f805075406ec7b68d/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:16.959155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5b5d8-8a10-4698-949e-63f35d8a1d7e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:16.959450 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5b5d8-8a10-4698-949e-63f35d8a1d7e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:16.959729 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5b5d8-8a10-4698-949e-63f35d8a1d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:16.975411 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f194f9-0eeb-4a6f-8b81-e5b391c83fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:16.976872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f194f9-0eeb-4a6f-8b81-e5b391c83fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:16.977169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f194f9-0eeb-4a6f-8b81-e5b391c83fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:16.977451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f194f9-0eeb-4a6f-8b81-e5b391c83fa0 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:16.978226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f194f9-0eeb-4a6f-8b81-e5b391c83fa0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:16.978537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f194f9-0eeb-4a6f-8b81-e5b391c83fa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:16.980354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f194f9-0eeb-4a6f-8b81-e5b391c83fa0 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:16.998156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 130/258] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:17.003836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f194f9-0eeb-4a6f-8b81-e5b391c83fa0 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:17.012306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-cae5b5d8-8a10-4698-949e-63f35d8a1d7e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:17.013578 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cae5b5d8-8a10-4698-949e-63f35d8a1d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:17.171918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f194f9-0eeb-4a6f-8b81-e5b391c83fa0 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:17.195409 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:17.237335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653087f-93d2-478c-982d-2de376e5e91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:17.238377 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653087f-93d2-478c-982d-2de376e5e91a None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:17.238664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653087f-93d2-478c-982d-2de376e5e91a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:17.238939 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653087f-93d2-478c-982d-2de376e5e91a None admin] PATH_INFO: `/v3/users/ae2439475b714dd1ba40dbe86303d708` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:17.240232 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653087f-93d2-478c-982d-2de376e5e91a None admin] RBAC: Authorizing `identity:get_user(user_id=ae2439475b714dd1ba40dbe86303d708)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:17.241859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653087f-93d2-478c-982d-2de376e5e91a None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:17.243890 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 131/259] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:17 2022] GET /identity/v3/users/ae2439475b714dd1ba40dbe86303d708 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:17.249343 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653087f-93d2-478c-982d-2de376e5e91a None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:17.349561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:17.354616 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 129/260] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:17.402693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5b5d8-8a10-4698-949e-63f35d8a1d7e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:17.454119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653087f-93d2-478c-982d-2de376e5e91a None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:17.530388 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476ea72-d74a-455a-be80-ba5395cc579c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:17.531060 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476ea72-d74a-455a-be80-ba5395cc579c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:17.531349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476ea72-d74a-455a-be80-ba5395cc579c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:17.531630 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476ea72-d74a-455a-be80-ba5395cc579c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:17.631440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5b5d8-8a10-4698-949e-63f35d8a1d7e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:17.709216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476ea72-d74a-455a-be80-ba5395cc579c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:17.709963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d2dfa-95df-403c-a0dc-222c6e310444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:17.710649 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476ea72-d74a-455a-be80-ba5395cc579c None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:17.710768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d2dfa-95df-403c-a0dc-222c6e310444 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:17.711059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d2dfa-95df-403c-a0dc-222c6e310444 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:17.711341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d2dfa-95df-403c-a0dc-222c6e310444 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:17.722895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d2dfa-95df-403c-a0dc-222c6e310444 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:17.731437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-3d9d2dfa-95df-403c-a0dc-222c6e310444 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:17.731437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:17.731437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:17.731437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:17.731437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:17.731437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:17.731437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:17.731437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:17.731437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:17.731437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:17.731437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:17.731437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:17.731437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:17.731437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:17.731437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:17.731437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:17.732359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:17.732359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:17.732359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:17.732359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:17.732359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:17.732359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:17.732359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:17.732359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:17.732359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:17.732359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:17.732359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:17.732359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:17.732359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:17.732359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:17.732359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:17.732359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:17.733170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:17.733170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:17.733170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:17.733170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:17.733170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:17.733170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:17.733170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:17.733170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:17.733170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:17.733170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:17.733170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:17.733170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:17.735573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 130/261] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:17.742124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d2dfa-95df-403c-a0dc-222c6e310444 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:17.808689 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:17.808880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:17.860894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d2dfa-95df-403c-a0dc-222c6e310444 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:17.896806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1011e73-9c2b-4c07-960b-871a80363a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:17.898398 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1011e73-9c2b-4c07-960b-871a80363a34 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:17.898902 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1011e73-9c2b-4c07-960b-871a80363a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:17.899487 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1011e73-9c2b-4c07-960b-871a80363a34 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:17.900635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1011e73-9c2b-4c07-960b-871a80363a34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:17.901166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1011e73-9c2b-4c07-960b-871a80363a34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:17.902930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1011e73-9c2b-4c07-960b-871a80363a34 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:17.913519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 131/262] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:17.919169 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1011e73-9c2b-4c07-960b-871a80363a34 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:17.980740 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 132/263] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 732 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:10:17.983454 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476ea72-d74a-455a-be80-ba5395cc579c None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:18.128704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476ea72-d74a-455a-be80-ba5395cc579c None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:18.132637 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:18.158263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1011e73-9c2b-4c07-960b-871a80363a34 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:18.189542 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ed0a8-723c-4584-9e70-ff6d56613c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:18.190420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ed0a8-723c-4584-9e70-ff6d56613c3f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.190704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ed0a8-723c-4584-9e70-ff6d56613c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.190982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ed0a8-723c-4584-9e70-ff6d56613c3f None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.193220 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ed0a8-723c-4584-9e70-ff6d56613c3f None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:18.195103 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ed0a8-723c-4584-9e70-ff6d56613c3f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:18.197172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 133/264] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:17 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:18.203168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ed0a8-723c-4584-9e70-ff6d56613c3f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:18.218722 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:18.269113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fad81-2b32-44e3-a7ea-954cc3bcc105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:18.269933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fad81-2b32-44e3-a7ea-954cc3bcc105 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.270227 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fad81-2b32-44e3-a7ea-954cc3bcc105 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.270531 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fad81-2b32-44e3-a7ea-954cc3bcc105 None admin] PATH_INFO: `/v3/users/ae2439475b714dd1ba40dbe86303d708` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.271793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997fad81-2b32-44e3-a7ea-954cc3bcc105 None admin] RBAC: Authorizing `identity:get_user(user_id=ae2439475b714dd1ba40dbe86303d708)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:18.273811 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997fad81-2b32-44e3-a7ea-954cc3bcc105 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:18.275792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 132/265] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:17 2022] GET /identity/v3/users/ae2439475b714dd1ba40dbe86303d708 => generated 283 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:18.281274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fad81-2b32-44e3-a7ea-954cc3bcc105 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:18.414864 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ed0a8-723c-4584-9e70-ff6d56613c3f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:18.471498 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fad81-2b32-44e3-a7ea-954cc3bcc105 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:18.497046 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc89e60-ae53-4293-b0cf-28b4f49db21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:18.497712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc89e60-ae53-4293-b0cf-28b4f49db21b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.497997 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc89e60-ae53-4293-b0cf-28b4f49db21b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.498273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc89e60-ae53-4293-b0cf-28b4f49db21b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.499132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc89e60-ae53-4293-b0cf-28b4f49db21b None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=359a065a7d2c49fea870a82587a5d51e, user.id=ae2439475b714dd1ba40dbe86303d708) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:18.499446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc89e60-ae53-4293-b0cf-28b4f49db21b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:18.500984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc89e60-ae53-4293-b0cf-28b4f49db21b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:18.514760 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 134/266] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:18 2022] GET /identity/v3/role_assignments?user.id=ae2439475b714dd1ba40dbe86303d708&scope.project.id=359a065a7d2c49fea870a82587a5d51e&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 608 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:18.525104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d8f8b-e6a4-4279-8183-f906eabecb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:18.525703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d8f8b-e6a4-4279-8183-f906eabecb01 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.525990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d8f8b-e6a4-4279-8183-f906eabecb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.526271 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d8f8b-e6a4-4279-8183-f906eabecb01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.573738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:18.626988 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d8f8b-e6a4-4279-8183-f906eabecb01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:18.628547 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d8f8b-e6a4-4279-8183-f906eabecb01 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:18.743480 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 133/267] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:18.749527 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc89e60-ae53-4293-b0cf-28b4f49db21b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:18.856693 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:18.869129 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a3566-ff55-486e-85e6-a8c1cee90958 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.869451 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a3566-ff55-486e-85e6-a8c1cee90958 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.869687 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a3566-ff55-486e-85e6-a8c1cee90958 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.872383 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 134/268] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:18.876914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c76d5-5a24-4003-8884-fa56cf1b0b61 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.877492 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c76d5-5a24-4003-8884-fa56cf1b0b61 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.878000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c76d5-5a24-4003-8884-fa56cf1b0b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.905473 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-795c76d5-5a24-4003-8884-fa56cf1b0b61 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:18.909071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-795c76d5-5a24-4003-8884-fa56cf1b0b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:18.914827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc89e60-ae53-4293-b0cf-28b4f49db21b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:18.977826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a3a51-b110-49d1-a855-435a1e1c985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:18.979075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a3a51-b110-49d1-a855-435a1e1c985a None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.979562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a3a51-b110-49d1-a855-435a1e1c985a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.980855 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a3a51-b110-49d1-a855-435a1e1c985a None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.986797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a3a51-b110-49d1-a855-435a1e1c985a None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:18.988514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a3a51-b110-49d1-a855-435a1e1c985a None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:18.993435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 135/269] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:18 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:18.997580 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a3a51-b110-49d1-a855-435a1e1c985a None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.018980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:19.036691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 135/270] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:19.093911 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c76d5-5a24-4003-8884-fa56cf1b0b61 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.138212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a3a51-b110-49d1-a855-435a1e1c985a None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:19.187712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f4be4-8429-429b-8cca-44c4a6b5c79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:19.191440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f4be4-8429-429b-8cca-44c4a6b5c79b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.191440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f4be4-8429-429b-8cca-44c4a6b5c79b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.191440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f4be4-8429-429b-8cca-44c4a6b5c79b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.191440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f4be4-8429-429b-8cca-44c4a6b5c79b None admin] RBAC: Adding query filter params (role.id=abe200f23fc84ba397dd8c67874f0f2a, scope.project.id=359a065a7d2c49fea870a82587a5d51e, user.id=ae2439475b714dd1ba40dbe86303d708) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:19.191440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f4be4-8429-429b-8cca-44c4a6b5c79b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:19.197127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f4be4-8429-429b-8cca-44c4a6b5c79b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:19.199794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 136/271] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:18 2022] GET /identity/v3/role_assignments?user.id=ae2439475b714dd1ba40dbe86303d708&scope.project.id=359a065a7d2c49fea870a82587a5d51e&role.id=abe200f23fc84ba397dd8c67874f0f2a => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:19.202875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e17478-a020-43d8-b630-96670d35757a None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.202875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e17478-a020-43d8-b630-96670d35757a None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.202875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e17478-a020-43d8-b630-96670d35757a None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.203123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 137/272] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:19.209668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f39cd-59bc-46a7-839a-a06bdccbf390 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.210525 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f39cd-59bc-46a7-839a-a06bdccbf390 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.210777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f39cd-59bc-46a7-839a-a06bdccbf390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.216505 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:19.230592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-689f39cd-59bc-46a7-839a-a06bdccbf390 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:19.232395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-689f39cd-59bc-46a7-839a-a06bdccbf390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:19.308705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 138/273] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:19.323276 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbd847-9dad-4378-a00f-d3f3dc6d8c1c None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.323566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbd847-9dad-4378-a00f-d3f3dc6d8c1c None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.323849 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbd847-9dad-4378-a00f-d3f3dc6d8c1c None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.325516 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 139/274] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:19.331874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f271b76-a327-4195-a93a-b95eaae33c87 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.332161 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f271b76-a327-4195-a93a-b95eaae33c87 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.332548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f271b76-a327-4195-a93a-b95eaae33c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.356935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-0f271b76-a327-4195-a93a-b95eaae33c87 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:19.357037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c76d5-5a24-4003-8884-fa56cf1b0b61 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:19.357127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0f271b76-a327-4195-a93a-b95eaae33c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:19.404261 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491318fb-3473-456e-9e82-59bfc08b52e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:19.405225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491318fb-3473-456e-9e82-59bfc08b52e1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.405610 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491318fb-3473-456e-9e82-59bfc08b52e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.405983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491318fb-3473-456e-9e82-59bfc08b52e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.421780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491318fb-3473-456e-9e82-59bfc08b52e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:19.433373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-491318fb-3473-456e-9e82-59bfc08b52e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:19.433373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:19.433373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:19.433373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:19.433373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:19.433373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:19.433373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:19.433373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:19.433373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:19.433373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:19.433373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:19.433373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:19.433373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:19.433373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:19.433373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:19.433373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:19.434572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:19.434572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:19.434572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:19.434572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:19.434572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:19.434572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:19.434572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:19.434572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:19.434572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:19.434572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:19.434572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:19.434572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:19.434572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:19.434572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:19.434572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:19.434572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:19.435360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:19.435360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:19.435360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:19.435360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:19.435360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:19.435360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:19.435360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:19.435360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:19.435360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:19.435360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:19.435360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:19.435360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:19.435960 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 136/275] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:19.437207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491318fb-3473-456e-9e82-59bfc08b52e1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.445012 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 140/276] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:19.446145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f271b76-a327-4195-a93a-b95eaae33c87 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.507819 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:19.532458 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:19.569975 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491318fb-3473-456e-9e82-59bfc08b52e1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:19.581847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f271b76-a327-4195-a93a-b95eaae33c87 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:19.605350 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78537b3b-78b5-4e5c-9485-10bd203c4497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:19.606150 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78537b3b-78b5-4e5c-9485-10bd203c4497 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.606439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78537b3b-78b5-4e5c-9485-10bd203c4497 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.606720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78537b3b-78b5-4e5c-9485-10bd203c4497 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.618276 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78537b3b-78b5-4e5c-9485-10bd203c4497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:19.620774 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778bee9d-5bf3-4d88-a8d7-2aca65501c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:19.621696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bee9d-5bf3-4d88-a8d7-2aca65501c13 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.622177 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bee9d-5bf3-4d88-a8d7-2aca65501c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.622647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bee9d-5bf3-4d88-a8d7-2aca65501c13 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.623592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778bee9d-5bf3-4d88-a8d7-2aca65501c13 None admin] RBAC: Adding query filter params (name=member) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:19.624095 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778bee9d-5bf3-4d88-a8d7-2aca65501c13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:19.625782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778bee9d-5bf3-4d88-a8d7-2aca65501c13 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:19.626148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-78537b3b-78b5-4e5c-9485-10bd203c4497 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:19.626148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:19.626148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:19.626148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:19.626148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:19.626148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:19.626148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:19.626148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:19.626148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:19.626148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:19.626148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:19.626148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:19.626148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:19.626148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:19.626148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:19.626148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:19.626997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:19.626997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:19.626997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:19.626997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:19.626997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:19.626997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:19.626997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:19.626997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:19.626997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:19.626997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:19.626997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:19.626997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:19.626997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:19.626997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:19.626997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:19.626997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:19.627771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:19.627771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:19.627771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:19.627771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:19.627771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:19.627771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:19.627771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:19.627771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:19.627771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:19.627771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:19.627771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:19.627771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:19.627771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 137/277] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:19.628515 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78537b3b-78b5-4e5c-9485-10bd203c4497 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.639937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 141/278] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:19.641074 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778bee9d-5bf3-4d88-a8d7-2aca65501c13 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.735988 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78537b3b-78b5-4e5c-9485-10bd203c4497 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:19.747976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778bee9d-5bf3-4d88-a8d7-2aca65501c13 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:19.749318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:19.774805 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:19.775486 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061291f-aed8-43e0-a6d8-9a331549ec2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:19.776303 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061291f-aed8-43e0-a6d8-9a331549ec2b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.776682 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061291f-aed8-43e0-a6d8-9a331549ec2b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.776903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061291f-aed8-43e0-a6d8-9a331549ec2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.778336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061291f-aed8-43e0-a6d8-9a331549ec2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:19.779792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061291f-aed8-43e0-a6d8-9a331549ec2b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:19.781462 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 138/279] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:19.782230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061291f-aed8-43e0-a6d8-9a331549ec2b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.792803 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed756c-d3ea-440e-b430-5bdf1b299632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:19.793729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed756c-d3ea-440e-b430-5bdf1b299632 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.794215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed756c-d3ea-440e-b430-5bdf1b299632 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.794668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed756c-d3ea-440e-b430-5bdf1b299632 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.795601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed756c-d3ea-440e-b430-5bdf1b299632 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:19.795914 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed756c-d3ea-440e-b430-5bdf1b299632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:19.797323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed756c-d3ea-440e-b430-5bdf1b299632 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:19.811112 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 142/280] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:19.811112 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed756c-d3ea-440e-b430-5bdf1b299632 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.885869 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061291f-aed8-43e0-a6d8-9a331549ec2b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:19.919612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed756c-d3ea-440e-b430-5bdf1b299632 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:19.923087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57743932-fb5b-48fb-ab78-231100e673cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:19.924047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57743932-fb5b-48fb-ab78-231100e673cd None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.924427 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57743932-fb5b-48fb-ab78-231100e673cd None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.924766 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57743932-fb5b-48fb-ab78-231100e673cd None admin] PATH_INFO: `/v3/users/56e073c25c354fd7a4bd22da3135baee` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.939573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57743932-fb5b-48fb-ab78-231100e673cd None admin] RBAC: Authorizing `identity:get_user(user_id=56e073c25c354fd7a4bd22da3135baee)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:19.941059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57743932-fb5b-48fb-ab78-231100e673cd None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:19.942756 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 139/281] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/users/56e073c25c354fd7a4bd22da3135baee => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:19.943519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57743932-fb5b-48fb-ab78-231100e673cd None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.960705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194850e-7c46-4f61-8acc-73795113761f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:19.961157 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194850e-7c46-4f61-8acc-73795113761f None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.961446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194850e-7c46-4f61-8acc-73795113761f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.961725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194850e-7c46-4f61-8acc-73795113761f None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.962445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194850e-7c46-4f61-8acc-73795113761f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:19.963880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194850e-7c46-4f61-8acc-73795113761f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:19.987515 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 143/282] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:10:19 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:19.987784 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:19.988264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194850e-7c46-4f61-8acc-73795113761f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.045668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:20.070808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57743932-fb5b-48fb-ab78-231100e673cd None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.111504 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f6a5b-1bb3-4ae1-b65f-2aba5022fef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.111728 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194850e-7c46-4f61-8acc-73795113761f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.112438 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f6a5b-1bb3-4ae1-b65f-2aba5022fef9 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.112727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f6a5b-1bb3-4ae1-b65f-2aba5022fef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.113006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f6a5b-1bb3-4ae1-b65f-2aba5022fef9 None admin] PATH_INFO: `/v3/users/a176881b0e804295849981bb76bd5391` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.114307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f6a5b-1bb3-4ae1-b65f-2aba5022fef9 None admin] RBAC: Authorizing `identity:get_user(user_id=a176881b0e804295849981bb76bd5391)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.115799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f6a5b-1bb3-4ae1-b65f-2aba5022fef9 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.122391 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 140/283] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/users/a176881b0e804295849981bb76bd5391 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:20.122391 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee62290-7893-492d-8301-e2aba67e90b5 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.122391 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee62290-7893-492d-8301-e2aba67e90b5 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.122391 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee62290-7893-492d-8301-e2aba67e90b5 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.122391 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 141/284] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:20.125997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee62290-7893-492d-8301-e2aba67e90b5 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.158742 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29514fbd-1700-4e8b-9d37-54f5b9d30e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.159738 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29514fbd-1700-4e8b-9d37-54f5b9d30e61 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.160257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29514fbd-1700-4e8b-9d37-54f5b9d30e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.160387 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29514fbd-1700-4e8b-9d37-54f5b9d30e61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.161177 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29514fbd-1700-4e8b-9d37-54f5b9d30e61 None admin] RBAC: Adding query filter params (role.id=18f0dc41b78c4b52b6912225928ea388, user.id=56e073c25c354fd7a4bd22da3135baee, scope.system=all) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:20.161473 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29514fbd-1700-4e8b-9d37-54f5b9d30e61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.163197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29514fbd-1700-4e8b-9d37-54f5b9d30e61 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.168612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 144/285] 10.222.0.43 () {56 vars in 1308 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/role_assignments?user.id=56e073c25c354fd7a4bd22da3135baee&scope.system=all&role.id=18f0dc41b78c4b52b6912225928ea388 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:20.169904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dae0c8-dd89-477d-8182-888315eadfd9 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.170196 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dae0c8-dd89-477d-8182-888315eadfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.170474 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dae0c8-dd89-477d-8182-888315eadfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.189184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-19dae0c8-dd89-477d-8182-888315eadfd9 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:20.190554 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-19dae0c8-dd89-477d-8182-888315eadfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:20.273450 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee62290-7893-492d-8301-e2aba67e90b5 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.282127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 145/286] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:20.283101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:20.314721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dae0c8-dd89-477d-8182-888315eadfd9 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.324474 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb911a1-3784-4a3c-99fd-d04b07b34f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.325600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb911a1-3784-4a3c-99fd-d04b07b34f5b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.325878 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb911a1-3784-4a3c-99fd-d04b07b34f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.326147 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb911a1-3784-4a3c-99fd-d04b07b34f5b None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.327402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb911a1-3784-4a3c-99fd-d04b07b34f5b None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.329048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb911a1-3784-4a3c-99fd-d04b07b34f5b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.332344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 142/287] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:20 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:20.337359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb911a1-3784-4a3c-99fd-d04b07b34f5b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.360328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:20.439435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dae0c8-dd89-477d-8182-888315eadfd9 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.475949 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb911a1-3784-4a3c-99fd-d04b07b34f5b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.478336 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d5782-701e-4d67-a155-783e22787ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.479148 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d5782-701e-4d67-a155-783e22787ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.479427 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d5782-701e-4d67-a155-783e22787ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.479702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d5782-701e-4d67-a155-783e22787ad2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.490575 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d5782-701e-4d67-a155-783e22787ad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:20.498453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-2b9d5782-701e-4d67-a155-783e22787ad2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:20.498453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:20.498453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:20.498453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:20.498453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:20.498453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:20.498453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:20.498453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:20.498453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:20.498453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:20.498453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:20.498453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:20.498453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:20.498453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:20.498453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:20.498453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:20.499349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:20.499349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:20.499349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:20.499349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:20.499349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:20.499349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:20.499349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:20.499349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:20.499349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:20.499349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:20.499349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:20.499349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:20.499349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:20.499349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:20.499349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:20.499349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:20.500133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:20.500133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:20.500133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:20.500133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:20.500133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:20.500133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:20.500133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:20.500133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:20.500133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:20.500133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:20.500133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:20.500133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:20.500133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 146/288] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:20.505900 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d5782-701e-4d67-a155-783e22787ad2 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.546117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee8ef8-f8bb-40c7-a168-8135dcd2a730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.546941 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee8ef8-f8bb-40c7-a168-8135dcd2a730 None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.547217 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee8ef8-f8bb-40c7-a168-8135dcd2a730 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.547488 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee8ef8-f8bb-40c7-a168-8135dcd2a730 None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e/users/a176881b0e804295849981bb76bd5391/roles/ba4641dd50c64f80a618b95aaaa321cb` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.554818 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee8ef8-f8bb-40c7-a168-8135dcd2a730 None admin] RBAC: Authorizing `identity:create_grant(project_id=359a065a7d2c49fea870a82587a5d51e, user_id=a176881b0e804295849981bb76bd5391, role_id=ba4641dd50c64f80a618b95aaaa321cb)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.556314 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee8ef8-f8bb-40c7-a168-8135dcd2a730 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.580601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 143/289] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:20 2022] PUT /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e/users/a176881b0e804295849981bb76bd5391/roles/ba4641dd50c64f80a618b95aaaa321cb => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:20.612006 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:20.619897 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d5782-701e-4d67-a155-783e22787ad2 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.672843 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9bcbb-bf0c-406b-ad9e-76ebee3053ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.673537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9bcbb-bf0c-406b-ad9e-76ebee3053ec None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.673822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9bcbb-bf0c-406b-ad9e-76ebee3053ec None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.674097 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9bcbb-bf0c-406b-ad9e-76ebee3053ec None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.674872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9bcbb-bf0c-406b-ad9e-76ebee3053ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:20.675183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9bcbb-bf0c-406b-ad9e-76ebee3053ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.676616 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9bcbb-bf0c-406b-ad9e-76ebee3053ec None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.692319 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 147/290] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:20.698259 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee8ef8-f8bb-40c7-a168-8135dcd2a730 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.737897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:20.816353 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee8ef8-f8bb-40c7-a168-8135dcd2a730 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.851079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6dba8-d986-469b-b840-2ddb130baa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.852385 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6dba8-d986-469b-b840-2ddb130baa28 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.852862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6dba8-d986-469b-b840-2ddb130baa28 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.853365 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6dba8-d986-469b-b840-2ddb130baa28 None admin] PATH_INFO: `/v3/users/b62462cb1ace4a2f805075406ec7b68d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.854766 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6dba8-d986-469b-b840-2ddb130baa28 None admin] RBAC: Authorizing `identity:get_user(user_id=b62462cb1ace4a2f805075406ec7b68d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.856450 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6dba8-d986-469b-b840-2ddb130baa28 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.858621 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 144/291] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:20 2022] GET /identity/v3/users/b62462cb1ace4a2f805075406ec7b68d => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:20.863892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9bcbb-bf0c-406b-ad9e-76ebee3053ec None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:21.017528 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9bcbb-bf0c-406b-ad9e-76ebee3053ec None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:21.069306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf958de8-5152-439b-8e05-534483ef6d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:21.069883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf958de8-5152-439b-8e05-534483ef6d8b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:21.070169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf958de8-5152-439b-8e05-534483ef6d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:21.070445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf958de8-5152-439b-8e05-534483ef6d8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:21.107324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:21.159931 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf958de8-5152-439b-8e05-534483ef6d8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:21.161428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf958de8-5152-439b-8e05-534483ef6d8b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:21.303676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 148/292] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:21.309194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6dba8-d986-469b-b840-2ddb130baa28 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:21.470871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6dba8-d986-469b-b840-2ddb130baa28 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:21.472464 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:21.524633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a2c6a-dfe1-473e-a22f-f8368b4d240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:21.525601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a2c6a-dfe1-473e-a22f-f8368b4d240c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:21.525887 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a2c6a-dfe1-473e-a22f-f8368b4d240c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:21.526167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a2c6a-dfe1-473e-a22f-f8368b4d240c None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:21.529338 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a2c6a-dfe1-473e-a22f-f8368b4d240c None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:21.530955 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a2c6a-dfe1-473e-a22f-f8368b4d240c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:21.533088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 145/293] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:21 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:21.539098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf958de8-5152-439b-8e05-534483ef6d8b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:21.634384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:21.688503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf958de8-5152-439b-8e05-534483ef6d8b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:21.724168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287efd42-6abb-475e-b970-304419aaa3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:21.724848 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287efd42-6abb-475e-b970-304419aaa3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:21.725134 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287efd42-6abb-475e-b970-304419aaa3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:21.725440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287efd42-6abb-475e-b970-304419aaa3b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:21.726277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287efd42-6abb-475e-b970-304419aaa3b6 None admin] RBAC: Adding query filter params (role.id=18f0dc41b78c4b52b6912225928ea388, scope.project.id=359a065a7d2c49fea870a82587a5d51e, user.id=b62462cb1ace4a2f805075406ec7b68d) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:21.726595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287efd42-6abb-475e-b970-304419aaa3b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:21.728028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287efd42-6abb-475e-b970-304419aaa3b6 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:21.734368 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 149/294] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:21 2022] GET /identity/v3/role_assignments?user.id=b62462cb1ace4a2f805075406ec7b68d&scope.project.id=359a065a7d2c49fea870a82587a5d51e&role.id=18f0dc41b78c4b52b6912225928ea388 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:22.604851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112cd84-7c48-4e1c-aca4-43b30ff65d46 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:22.605337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112cd84-7c48-4e1c-aca4-43b30ff65d46 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:22.605337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112cd84-7c48-4e1c-aca4-43b30ff65d46 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:22.606942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 146/295] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:22.612327 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bc505-7f0c-4204-adeb-c9519aa87142 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:22.612846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bc505-7f0c-4204-adeb-c9519aa87142 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:22.612999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bc505-7f0c-4204-adeb-c9519aa87142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:22.630583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-f92bc505-7f0c-4204-adeb-c9519aa87142 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:22.631793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f92bc505-7f0c-4204-adeb-c9519aa87142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:22.677894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f0f80-3f0a-4f18-8306-5a2c96f7600a None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:22.678232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f0f80-3f0a-4f18-8306-5a2c96f7600a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:22.679020 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f0f80-3f0a-4f18-8306-5a2c96f7600a None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:22.680685 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 147/296] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:22.686343 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36b254-f229-4fcf-8518-cf2aa5b623a4 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:22.686872 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36b254-f229-4fcf-8518-cf2aa5b623a4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:22.687363 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36b254-f229-4fcf-8518-cf2aa5b623a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:22.704880 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-cf36b254-f229-4fcf-8518-cf2aa5b623a4 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:22.706465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cf36b254-f229-4fcf-8518-cf2aa5b623a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:22.709865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 150/297] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:22.723936 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea4332-73c1-407d-9760-5a5f5780dcc5 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:22.724253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea4332-73c1-407d-9760-5a5f5780dcc5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:22.724544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea4332-73c1-407d-9760-5a5f5780dcc5 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:22.726903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 151/298] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:22.732367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdb2d3-7b78-4ca9-9088-71a725f60e8d None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:22.732647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdb2d3-7b78-4ca9-9088-71a725f60e8d None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:22.732918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdb2d3-7b78-4ca9-9088-71a725f60e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:22.749895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-acbdb2d3-7b78-4ca9-9088-71a725f60e8d None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:22.751102 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-acbdb2d3-7b78-4ca9-9088-71a725f60e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:22.779989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 148/299] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:22.781287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36b254-f229-4fcf-8518-cf2aa5b623a4 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:22.813950 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:22.836510 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 152/300] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:22.841558 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdb2d3-7b78-4ca9-9088-71a725f60e8d None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:22.886097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:22.889190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36b254-f229-4fcf-8518-cf2aa5b623a4 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:22.928858 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2f739-d046-4476-8d70-120df818121b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:22.929653 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2f739-d046-4476-8d70-120df818121b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:22.929931 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2f739-d046-4476-8d70-120df818121b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:22.930201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2f739-d046-4476-8d70-120df818121b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:22.941183 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2f739-d046-4476-8d70-120df818121b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:22.948874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-fde2f739-d046-4476-8d70-120df818121b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:22.948874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:22.948874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:22.948874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:22.948874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:22.948874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:22.948874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:22.948874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:22.948874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:22.948874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:22.948874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:22.948874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:22.948874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:22.948874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:22.948874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:22.948874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:22.949773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:22.949773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:22.949773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:22.949773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:22.949773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:22.949773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:22.949773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:22.949773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:22.949773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:22.949773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:22.949773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:22.949773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:22.949773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:22.949773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:22.949773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:22.949773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:22.950568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:22.950568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:22.950568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:22.950568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:22.950568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:22.950568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:22.950568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:22.950568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:22.950568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:22.950568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:22.950568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:22.950568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:22.950568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 149/301] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:22.951207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2f739-d046-4476-8d70-120df818121b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:22.967841 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdb2d3-7b78-4ca9-9088-71a725f60e8d None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.004463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a56a47-0341-4532-846c-aa7ac1a0a68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.005575 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a56a47-0341-4532-846c-aa7ac1a0a68b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.006047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a56a47-0341-4532-846c-aa7ac1a0a68b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.006320 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a56a47-0341-4532-846c-aa7ac1a0a68b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.007534 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a56a47-0341-4532-846c-aa7ac1a0a68b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:23.008993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a56a47-0341-4532-846c-aa7ac1a0a68b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:23.010609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 153/302] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:23.011366 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a56a47-0341-4532-846c-aa7ac1a0a68b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:23.061748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2f739-d046-4476-8d70-120df818121b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.066762 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:23.096393 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed39f32-386d-429e-bfc9-b3279aecf4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.097183 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed39f32-386d-429e-bfc9-b3279aecf4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.097465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed39f32-386d-429e-bfc9-b3279aecf4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.097740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed39f32-386d-429e-bfc9-b3279aecf4a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.108482 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed39f32-386d-429e-bfc9-b3279aecf4a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:23.116175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-1ed39f32-386d-429e-bfc9-b3279aecf4a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:23.116175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:23.116175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:23.116175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:23.116175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:23.116175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:23.116175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:23.116175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:23.116175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:23.116175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:23.116175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:23.116175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:23.116175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:23.116175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:23.116175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:23.116175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:23.117074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:23.117074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:23.117074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:23.117074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:23.117074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:23.117074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:23.117074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:23.117074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:23.117074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:23.117074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:23.117074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:23.117074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:23.117074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:23.117074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:23.117074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:23.117074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:23.122921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:23.122921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:23.122921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:23.122921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:23.122921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:23.122921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:23.122921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:23.122921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:23.122921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:23.122921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:23.122921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:23.122921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:23.123670 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a56a47-0341-4532-846c-aa7ac1a0a68b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.123753 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 150/303] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:23.123753 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed39f32-386d-429e-bfc9-b3279aecf4a2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:23.140927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:23.159825 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5182f32-a2fc-4f39-a912-a50cd5617489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.160530 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5182f32-a2fc-4f39-a912-a50cd5617489 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.160810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5182f32-a2fc-4f39-a912-a50cd5617489 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.161082 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5182f32-a2fc-4f39-a912-a50cd5617489 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.161826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5182f32-a2fc-4f39-a912-a50cd5617489 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:23.162137 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5182f32-a2fc-4f39-a912-a50cd5617489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:23.163538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5182f32-a2fc-4f39-a912-a50cd5617489 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:23.171417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 154/304] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:23.171417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49250f9a-94a4-46d7-ab1b-92254fd4d91b None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.171417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49250f9a-94a4-46d7-ab1b-92254fd4d91b None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.171417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49250f9a-94a4-46d7-ab1b-92254fd4d91b None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.171417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 155/305] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:23.171417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49250f9a-94a4-46d7-ab1b-92254fd4d91b None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:23.221567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed39f32-386d-429e-bfc9-b3279aecf4a2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.253549 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af6b17-987f-4647-bd87-aff42c1dc6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.254218 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af6b17-987f-4647-bd87-aff42c1dc6f9 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.254512 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af6b17-987f-4647-bd87-aff42c1dc6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.254786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af6b17-987f-4647-bd87-aff42c1dc6f9 None admin] PATH_INFO: `/v3/groups` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.255477 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af6b17-987f-4647-bd87-aff42c1dc6f9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:23.257488 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af6b17-987f-4647-bd87-aff42c1dc6f9 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:23.274198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 151/306] 10.222.0.43 () {60 vars in 1134 bytes} [Wed Jul 20 19:10:23 2022] POST /identity/v3/groups => generated 229 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:23.275257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af6b17-987f-4647-bd87-aff42c1dc6f9 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:23.279806 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49250f9a-94a4-46d7-ab1b-92254fd4d91b None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.296503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:23.323920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586efc1-a386-4f83-84b8-5f9f3d7b7057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.324674 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586efc1-a386-4f83-84b8-5f9f3d7b7057 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.324958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586efc1-a386-4f83-84b8-5f9f3d7b7057 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.325240 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586efc1-a386-4f83-84b8-5f9f3d7b7057 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.326018 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586efc1-a386-4f83-84b8-5f9f3d7b7057 None admin] RBAC: Adding query filter params (name=member) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:23.326329 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586efc1-a386-4f83-84b8-5f9f3d7b7057 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:23.327762 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586efc1-a386-4f83-84b8-5f9f3d7b7057 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:23.341298 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 156/307] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:23.342530 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e39df-46ab-4992-9401-7ff185e65db0 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.342820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e39df-46ab-4992-9401-7ff185e65db0 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.343098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e39df-46ab-4992-9401-7ff185e65db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.360716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-c02e39df-46ab-4992-9401-7ff185e65db0 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:23.361946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c02e39df-46ab-4992-9401-7ff185e65db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:23.394543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:23.417456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af6b17-987f-4647-bd87-aff42c1dc6f9 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.442789 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 157/308] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:23.444943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e39df-46ab-4992-9401-7ff185e65db0 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:23.457947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af0e65-23fb-41e5-808d-73ba490180d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.459252 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af0e65-23fb-41e5-808d-73ba490180d4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.460406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af0e65-23fb-41e5-808d-73ba490180d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.460837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af0e65-23fb-41e5-808d-73ba490180d4 None admin] PATH_INFO: `/v3/users/d5fa4d4a59de4ab9b57d76bb7acd4278` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.479349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af0e65-23fb-41e5-808d-73ba490180d4 None admin] RBAC: Authorizing `identity:get_user(user_id=d5fa4d4a59de4ab9b57d76bb7acd4278)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:23.480851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af0e65-23fb-41e5-808d-73ba490180d4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:23.482578 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 152/309] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:23 2022] GET /identity/v3/users/d5fa4d4a59de4ab9b57d76bb7acd4278 => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:23.483345 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af0e65-23fb-41e5-808d-73ba490180d4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:23.560005 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e39df-46ab-4992-9401-7ff185e65db0 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.563192 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:23.595123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af0e65-23fb-41e5-808d-73ba490180d4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.599318 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d417e5-456b-4804-b55b-6318a8e6f174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.600228 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d417e5-456b-4804-b55b-6318a8e6f174 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.600544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d417e5-456b-4804-b55b-6318a8e6f174 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.600818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d417e5-456b-4804-b55b-6318a8e6f174 None admin] PATH_INFO: `/v3/users/56e073c25c354fd7a4bd22da3135baee` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.602048 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d417e5-456b-4804-b55b-6318a8e6f174 None admin] RBAC: Authorizing `identity:get_user(user_id=56e073c25c354fd7a4bd22da3135baee)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:23.603496 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d417e5-456b-4804-b55b-6318a8e6f174 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:23.609725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 158/310] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:23 2022] GET /identity/v3/users/56e073c25c354fd7a4bd22da3135baee => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:23.609725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d417e5-456b-4804-b55b-6318a8e6f174 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:23.635687 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12e077-5562-4572-92bd-f823275630f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.636513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12e077-5562-4572-92bd-f823275630f4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.636790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12e077-5562-4572-92bd-f823275630f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.637063 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12e077-5562-4572-92bd-f823275630f4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.644464 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:23.649021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12e077-5562-4572-92bd-f823275630f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:23.657111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-5e12e077-5562-4572-92bd-f823275630f4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:23.657111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:23.657111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:23.657111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:23.657111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:23.657111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:23.657111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:23.657111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:23.657111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:23.657111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:23.657111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:23.657111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:23.657111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:23.657111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:23.657111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:23.657111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:23.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:23.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:23.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:23.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:23.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:23.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:23.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:23.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:23.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:23.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:23.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:23.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:23.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:23.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:23.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:23.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:23.658773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:23.658773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:23.658773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:23.658773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:23.658773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:23.658773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:23.658773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:23.658773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:23.658773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:23.658773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:23.658773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:23.658773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:23.658773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 153/311] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:23.659424 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12e077-5562-4572-92bd-f823275630f4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:23.723171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d417e5-456b-4804-b55b-6318a8e6f174 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.759574 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a952b-9bb3-4d12-8c07-f199e3283ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.760234 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a952b-9bb3-4d12-8c07-f199e3283ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.760547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a952b-9bb3-4d12-8c07-f199e3283ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.760824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a952b-9bb3-4d12-8c07-f199e3283ae9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.761660 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a952b-9bb3-4d12-8c07-f199e3283ae9 None admin] RBAC: Adding query filter params (role.id=ba4641dd50c64f80a618b95aaaa321cb, user.id=d5fa4d4a59de4ab9b57d76bb7acd4278, scope.system=all) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:23.761975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a952b-9bb3-4d12-8c07-f199e3283ae9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:23.763397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a952b-9bb3-4d12-8c07-f199e3283ae9 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:23.768466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 159/312] 10.222.0.43 () {56 vars in 1308 bytes} [Wed Jul 20 19:10:23 2022] GET /identity/v3/role_assignments?user.id=d5fa4d4a59de4ab9b57d76bb7acd4278&scope.system=all&role.id=ba4641dd50c64f80a618b95aaaa321cb => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:23.769244 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a952b-9bb3-4d12-8c07-f199e3283ae9 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:23.791863 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12e077-5562-4572-92bd-f823275630f4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.811745 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:23.856399 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3f425-530a-43bc-9b26-aecd6940e28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.857731 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3f425-530a-43bc-9b26-aecd6940e28b None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.858225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3f425-530a-43bc-9b26-aecd6940e28b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.858689 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3f425-530a-43bc-9b26-aecd6940e28b None admin] PATH_INFO: `/v3/system/users/56e073c25c354fd7a4bd22da3135baee/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.861527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3f425-530a-43bc-9b26-aecd6940e28b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=56e073c25c354fd7a4bd22da3135baee, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:23.863866 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3f425-530a-43bc-9b26-aecd6940e28b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:23.877868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 154/313] 10.222.0.43 () {58 vars in 1334 bytes} [Wed Jul 20 19:10:23 2022] PUT /identity/v3/system/users/56e073c25c354fd7a4bd22da3135baee/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:23.906051 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a952b-9bb3-4d12-8c07-f199e3283ae9 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.948682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2d941-6593-4bc6-8cc5-405df5d025ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.949386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2d941-6593-4bc6-8cc5-405df5d025ed None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.949653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2d941-6593-4bc6-8cc5-405df5d025ed None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.949927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2d941-6593-4bc6-8cc5-405df5d025ed None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.950699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2d941-6593-4bc6-8cc5-405df5d025ed None admin] RBAC: Adding query filter params (name=reader) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:23.951059 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2d941-6593-4bc6-8cc5-405df5d025ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:23.952496 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2d941-6593-4bc6-8cc5-405df5d025ed None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:23.962754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 160/314] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:23.968642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3f425-530a-43bc-9b26-aecd6940e28b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:24.044409 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:24.086837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3f425-530a-43bc-9b26-aecd6940e28b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:24.122388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d07d88-ff01-44fc-8cd5-5253bbd92b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:24.123261 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d07d88-ff01-44fc-8cd5-5253bbd92b72 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:24.123545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d07d88-ff01-44fc-8cd5-5253bbd92b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:24.123821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d07d88-ff01-44fc-8cd5-5253bbd92b72 None admin] PATH_INFO: `/v3/users/a176881b0e804295849981bb76bd5391` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:24.125047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d07d88-ff01-44fc-8cd5-5253bbd92b72 None admin] RBAC: Authorizing `identity:get_user(user_id=a176881b0e804295849981bb76bd5391)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:24.126512 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d07d88-ff01-44fc-8cd5-5253bbd92b72 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:24.128493 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 155/315] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:23 2022] GET /identity/v3/users/a176881b0e804295849981bb76bd5391 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:24.133954 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2d941-6593-4bc6-8cc5-405df5d025ed None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:24.236996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:24.243752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2d941-6593-4bc6-8cc5-405df5d025ed None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:24.279663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08179e22-115c-4c01-a641-09aa581f5f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:24.280732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08179e22-115c-4c01-a641-09aa581f5f4b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:24.281245 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08179e22-115c-4c01-a641-09aa581f5f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:24.281722 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08179e22-115c-4c01-a641-09aa581f5f4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:24.356511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08179e22-115c-4c01-a641-09aa581f5f4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:24.357985 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08179e22-115c-4c01-a641-09aa581f5f4b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:24.462950 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:24.473314 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 161/316] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:24.475263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d07d88-ff01-44fc-8cd5-5253bbd92b72 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:24.578815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d07d88-ff01-44fc-8cd5-5253bbd92b72 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:24.609139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:24.615240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da9035-e0a6-4780-b3ee-fb694587ec77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:24.616133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da9035-e0a6-4780-b3ee-fb694587ec77 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:24.616459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da9035-e0a6-4780-b3ee-fb694587ec77 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:24.616736 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da9035-e0a6-4780-b3ee-fb694587ec77 None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:24.618313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da9035-e0a6-4780-b3ee-fb694587ec77 None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:24.620148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da9035-e0a6-4780-b3ee-fb694587ec77 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:24.622219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 156/317] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:24 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:24.627964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08179e22-115c-4c01-a641-09aa581f5f4b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:24.731868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08179e22-115c-4c01-a641-09aa581f5f4b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:24.768561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd32d7-c164-4550-8345-5aea4fa0188d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:24.769210 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd32d7-c164-4550-8345-5aea4fa0188d None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:24.769490 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd32d7-c164-4550-8345-5aea4fa0188d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:24.769763 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd32d7-c164-4550-8345-5aea4fa0188d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:24.770594 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd32d7-c164-4550-8345-5aea4fa0188d None admin] RBAC: Adding query filter params (role.id=ba4641dd50c64f80a618b95aaaa321cb, scope.project.id=359a065a7d2c49fea870a82587a5d51e, user.id=a176881b0e804295849981bb76bd5391) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:24.770902 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd32d7-c164-4550-8345-5aea4fa0188d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:24.772907 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd32d7-c164-4550-8345-5aea4fa0188d None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:24.779164 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 162/318] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:24 2022] GET /identity/v3/role_assignments?user.id=a176881b0e804295849981bb76bd5391&scope.project.id=359a065a7d2c49fea870a82587a5d51e&role.id=ba4641dd50c64f80a618b95aaaa321cb => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:25.373048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc2ae8-f30c-4472-9625-7b0d69869a76 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.373509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc2ae8-f30c-4472-9625-7b0d69869a76 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.373608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc2ae8-f30c-4472-9625-7b0d69869a76 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.375181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 157/319] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:25.380679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278a1f1-a26b-466c-8b61-05eec8b6da09 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.380977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278a1f1-a26b-466c-8b61-05eec8b6da09 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.381263 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278a1f1-a26b-466c-8b61-05eec8b6da09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.398591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-3278a1f1-a26b-466c-8b61-05eec8b6da09 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:25.399789 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-3278a1f1-a26b-466c-8b61-05eec8b6da09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:25.458936 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:25.471195 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 163/320] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:25.497356 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc2ae8-f30c-4472-9625-7b0d69869a76 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:25.610575 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc2ae8-f30c-4472-9625-7b0d69869a76 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:25.644725 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4784905-7452-4cab-a8d9-f2db01254015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:25.645530 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4784905-7452-4cab-a8d9-f2db01254015 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.645822 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4784905-7452-4cab-a8d9-f2db01254015 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.646106 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4784905-7452-4cab-a8d9-f2db01254015 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.647323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4784905-7452-4cab-a8d9-f2db01254015 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:25.648817 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4784905-7452-4cab-a8d9-f2db01254015 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:25.652363 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 158/321] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:25.656251 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278a1f1-a26b-466c-8b61-05eec8b6da09 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:25.757562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278a1f1-a26b-466c-8b61-05eec8b6da09 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:25.793823 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8570f94-7d74-41eb-a1d8-5361f9b6472f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:25.794501 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8570f94-7d74-41eb-a1d8-5361f9b6472f None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.794788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8570f94-7d74-41eb-a1d8-5361f9b6472f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.795068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8570f94-7d74-41eb-a1d8-5361f9b6472f None admin] PATH_INFO: `/v3/groups` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.795774 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8570f94-7d74-41eb-a1d8-5361f9b6472f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:25.797235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8570f94-7d74-41eb-a1d8-5361f9b6472f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:25.820317 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 164/322] 10.222.0.43 () {60 vars in 1134 bytes} [Wed Jul 20 19:10:25 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:10:25.886527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545704a1-ef88-4163-857c-4e0942e3a639 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.886767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545704a1-ef88-4163-857c-4e0942e3a639 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.887014 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545704a1-ef88-4163-857c-4e0942e3a639 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.888608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 159/323] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:25.894154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e6d4d-1134-43de-b4ff-9ddf576caea3 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.894443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e6d4d-1134-43de-b4ff-9ddf576caea3 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.894718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e6d4d-1134-43de-b4ff-9ddf576caea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.912771 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-675e6d4d-1134-43de-b4ff-9ddf576caea3 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:25.914463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-675e6d4d-1134-43de-b4ff-9ddf576caea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:25.966673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99269ae4-d7cc-42e3-8925-de42425240cf None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.966673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99269ae4-d7cc-42e3-8925-de42425240cf None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.966971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99269ae4-d7cc-42e3-8925-de42425240cf None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.968372 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:25.968675 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 160/324] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:25.974401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccda6d8-2bfb-483a-bfd7-c1e63a47f749 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.974687 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccda6d8-2bfb-483a-bfd7-c1e63a47f749 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.974973 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccda6d8-2bfb-483a-bfd7-c1e63a47f749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.993573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-bccda6d8-2bfb-483a-bfd7-c1e63a47f749 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:25.994763 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bccda6d8-2bfb-483a-bfd7-c1e63a47f749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:25.998484 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 165/325] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:26.020279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e6d4d-1134-43de-b4ff-9ddf576caea3 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.075539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:26.087092 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 161/326] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:26.114968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccda6d8-2bfb-483a-bfd7-c1e63a47f749 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.145498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e6d4d-1134-43de-b4ff-9ddf576caea3 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.189847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428e7b8-1bf6-4bb3-9e91-f56214479b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.190653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428e7b8-1bf6-4bb3-9e91-f56214479b89 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.190933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428e7b8-1bf6-4bb3-9e91-f56214479b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.191206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428e7b8-1bf6-4bb3-9e91-f56214479b89 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.203814 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428e7b8-1bf6-4bb3-9e91-f56214479b89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:26.213669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-1428e7b8-1bf6-4bb3-9e91-f56214479b89 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:26.213669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:26.213669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:26.213669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:26.213669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:26.213669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:26.213669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:26.213669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:26.213669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:26.213669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:26.213669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:26.213669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:26.213669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:26.213669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:26.213669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:26.213669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:26.214578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:26.214578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:26.214578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:26.214578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:26.214578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:26.214578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:26.214578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:26.214578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:26.214578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:26.214578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:26.214578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:26.214578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:26.214578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:26.214578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:26.214578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:26.214578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:26.215767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:26.215767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:26.215767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:26.215767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:26.215767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:26.215767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:26.215767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:26.215767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:26.215767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:26.215767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:26.215767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:26.215767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:26.215767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 166/327] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:26.222832 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428e7b8-1bf6-4bb3-9e91-f56214479b89 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.243337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccda6d8-2bfb-483a-bfd7-c1e63a47f749 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.283660 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:26.291669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9395315-7621-452f-bf55-ec8468de3f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.293208 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9395315-7621-452f-bf55-ec8468de3f45 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.293690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9395315-7621-452f-bf55-ec8468de3f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.294154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9395315-7621-452f-bf55-ec8468de3f45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.311049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9395315-7621-452f-bf55-ec8468de3f45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:26.322612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-a9395315-7621-452f-bf55-ec8468de3f45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:26.322612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:26.322612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:26.322612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:26.322612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:26.322612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:26.322612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:26.322612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:26.322612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:26.322612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:26.322612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:26.322612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:26.322612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:26.322612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:26.322612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:26.322612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:26.323507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:26.323507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:26.323507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:26.323507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:26.323507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:26.323507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:26.323507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:26.323507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:26.323507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:26.323507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:26.323507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:26.323507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:26.323507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:26.323507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:26.323507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:26.323507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:26.324274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:26.324274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:26.324274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:26.324274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:26.324274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:26.324274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:26.324274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:26.324274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:26.324274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:26.324274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:26.324274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:26.324274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:26.327652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 162/328] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:26.333292 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9395315-7621-452f-bf55-ec8468de3f45 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.369474 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428e7b8-1bf6-4bb3-9e91-f56214479b89 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.404648 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:26.422727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028069b6-babd-478f-815d-251313b9ae5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.423765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028069b6-babd-478f-815d-251313b9ae5f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.424245 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028069b6-babd-478f-815d-251313b9ae5f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.424777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028069b6-babd-478f-815d-251313b9ae5f None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.425745 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028069b6-babd-478f-815d-251313b9ae5f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:26.426768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028069b6-babd-478f-815d-251313b9ae5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:26.428898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028069b6-babd-478f-815d-251313b9ae5f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:26.441020 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 167/329] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:26.449415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028069b6-babd-478f-815d-251313b9ae5f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.464169 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9395315-7621-452f-bf55-ec8468de3f45 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.505589 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1821941-e9f7-4668-9812-75e29c98a0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.506305 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1821941-e9f7-4668-9812-75e29c98a0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.507240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1821941-e9f7-4668-9812-75e29c98a0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.507520 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1821941-e9f7-4668-9812-75e29c98a0e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.508313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1821941-e9f7-4668-9812-75e29c98a0e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:26.509833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1821941-e9f7-4668-9812-75e29c98a0e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:26.511471 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1821941-e9f7-4668-9812-75e29c98a0e3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:26.521218 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 163/330] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:26.527446 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1821941-e9f7-4668-9812-75e29c98a0e3 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.585532 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028069b6-babd-478f-815d-251313b9ae5f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.589453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:26.639503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06b5c6-64ef-4579-8632-d2d6fe0ea8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.640793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06b5c6-64ef-4579-8632-d2d6fe0ea8af None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.641283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06b5c6-64ef-4579-8632-d2d6fe0ea8af None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.641865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06b5c6-64ef-4579-8632-d2d6fe0ea8af None admin] PATH_INFO: `/v3/users/d5fa4d4a59de4ab9b57d76bb7acd4278` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.643469 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06b5c6-64ef-4579-8632-d2d6fe0ea8af None admin] RBAC: Authorizing `identity:get_user(user_id=d5fa4d4a59de4ab9b57d76bb7acd4278)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:26.645066 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06b5c6-64ef-4579-8632-d2d6fe0ea8af None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:26.647139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 168/331] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:26 2022] GET /identity/v3/users/d5fa4d4a59de4ab9b57d76bb7acd4278 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:26.654954 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06b5c6-64ef-4579-8632-d2d6fe0ea8af None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.671975 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1821941-e9f7-4668-9812-75e29c98a0e3 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.697599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:26.721235 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80919d6-6d13-47e6-867d-0a8b6c4d92e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.722127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80919d6-6d13-47e6-867d-0a8b6c4d92e0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.722579 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80919d6-6d13-47e6-867d-0a8b6c4d92e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.722995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80919d6-6d13-47e6-867d-0a8b6c4d92e0 None admin] PATH_INFO: `/v3/users/56e073c25c354fd7a4bd22da3135baee` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.724559 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80919d6-6d13-47e6-867d-0a8b6c4d92e0 None admin] RBAC: Authorizing `identity:get_user(user_id=56e073c25c354fd7a4bd22da3135baee)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:26.726483 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80919d6-6d13-47e6-867d-0a8b6c4d92e0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:26.728595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 164/332] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:26 2022] GET /identity/v3/users/56e073c25c354fd7a4bd22da3135baee => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:26.734607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80919d6-6d13-47e6-867d-0a8b6c4d92e0 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.796397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06b5c6-64ef-4579-8632-d2d6fe0ea8af None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.838312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a44694-a22e-4c8b-8059-1f5ad0525551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.839580 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a44694-a22e-4c8b-8059-1f5ad0525551 None admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.840067 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a44694-a22e-4c8b-8059-1f5ad0525551 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.840594 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a44694-a22e-4c8b-8059-1f5ad0525551 None admin] PATH_INFO: `/v3/system/users/d5fa4d4a59de4ab9b57d76bb7acd4278/roles/ba4641dd50c64f80a618b95aaaa321cb` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.842338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a44694-a22e-4c8b-8059-1f5ad0525551 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d5fa4d4a59de4ab9b57d76bb7acd4278, role_id=ba4641dd50c64f80a618b95aaaa321cb)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:26.844794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a44694-a22e-4c8b-8059-1f5ad0525551 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:26.860754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 169/333] 10.222.0.43 () {58 vars in 1334 bytes} [Wed Jul 20 19:10:26 2022] PUT /identity/v3/system/users/d5fa4d4a59de4ab9b57d76bb7acd4278/roles/ba4641dd50c64f80a618b95aaaa321cb => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:26.878000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80919d6-6d13-47e6-867d-0a8b6c4d92e0 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.941215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a2f4e-a516-4c70-b4f6-1b485847f37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.941814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a2f4e-a516-4c70-b4f6-1b485847f37e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.942094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a2f4e-a516-4c70-b4f6-1b485847f37e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.942364 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a2f4e-a516-4c70-b4f6-1b485847f37e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.943220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a2f4e-a516-4c70-b4f6-1b485847f37e None admin] RBAC: Adding query filter params (role.id=18f0dc41b78c4b52b6912225928ea388, user.id=56e073c25c354fd7a4bd22da3135baee, scope.system=all) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:26.943531 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a2f4e-a516-4c70-b4f6-1b485847f37e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:26.945100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a2f4e-a516-4c70-b4f6-1b485847f37e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:26.959261 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 165/334] 10.222.0.43 () {56 vars in 1308 bytes} [Wed Jul 20 19:10:26 2022] GET /identity/v3/role_assignments?user.id=56e073c25c354fd7a4bd22da3135baee&scope.system=all&role.id=18f0dc41b78c4b52b6912225928ea388 => generated 510 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:28.210159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208b900-6bba-4fc5-8a29-cd7c728898ad None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.210676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208b900-6bba-4fc5-8a29-cd7c728898ad None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.210784 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208b900-6bba-4fc5-8a29-cd7c728898ad None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.212426 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 170/335] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:28.217953 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935761b-f3af-485e-bbb5-b8922f0545e4 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.218369 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935761b-f3af-485e-bbb5-b8922f0545e4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.218555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935761b-f3af-485e-bbb5-b8922f0545e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.238857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f935761b-f3af-485e-bbb5-b8922f0545e4 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:28.240068 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f935761b-f3af-485e-bbb5-b8922f0545e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:28.294314 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:28.326563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 166/336] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:28.361219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208b900-6bba-4fc5-8a29-cd7c728898ad None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:28.416338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:28.500229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208b900-6bba-4fc5-8a29-cd7c728898ad None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:28.552840 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce30411-f170-4e85-b5b5-28217de63a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:28.554205 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce30411-f170-4e85-b5b5-28217de63a1c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.554491 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce30411-f170-4e85-b5b5-28217de63a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.554767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce30411-f170-4e85-b5b5-28217de63a1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.572850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce30411-f170-4e85-b5b5-28217de63a1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:28.580710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-dce30411-f170-4e85-b5b5-28217de63a1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:28.580710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:28.580710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:28.580710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:28.580710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:28.580710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:28.580710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:28.580710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:28.580710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:28.580710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:28.580710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:28.580710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:28.580710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:28.580710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:28.580710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:28.580710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:28.581595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:28.581595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:28.581595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:28.581595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:28.581595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:28.581595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:28.581595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:28.581595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:28.581595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:28.581595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:28.581595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:28.581595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:28.581595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:28.581595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:28.581595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:28.581595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:28.582435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:28.582435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:28.582435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:28.582435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:28.582435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:28.582435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:28.582435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:28.582435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:28.582435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:28.582435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:28.582435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:28.582435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:28.583482 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 171/337] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:28.590321 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935761b-f3af-485e-bbb5-b8922f0545e4 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:28.677829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85f4cf-4dac-484a-9a8b-1ccf2bf07d16 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.678215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85f4cf-4dac-484a-9a8b-1ccf2bf07d16 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.678958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85f4cf-4dac-484a-9a8b-1ccf2bf07d16 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.684532 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 172/338] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:28.686200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfd399-8be0-4f4b-8dc2-194f5eef5fa4 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.686610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfd399-8be0-4f4b-8dc2-194f5eef5fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.686889 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfd399-8be0-4f4b-8dc2-194f5eef5fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.711138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935761b-f3af-485e-bbb5-b8922f0545e4 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:28.719378 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-59bfd399-8be0-4f4b-8dc2-194f5eef5fa4 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:28.720698 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-59bfd399-8be0-4f4b-8dc2-194f5eef5fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:28.747520 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca8702-6a9f-49d3-9ef2-6ee34a5b6169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:28.748190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca8702-6a9f-49d3-9ef2-6ee34a5b6169 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.748518 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca8702-6a9f-49d3-9ef2-6ee34a5b6169 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.748798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca8702-6a9f-49d3-9ef2-6ee34a5b6169 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.749685 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca8702-6a9f-49d3-9ef2-6ee34a5b6169 None admin] RBAC: Adding query filter params (name=member) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:28.749996 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca8702-6a9f-49d3-9ef2-6ee34a5b6169 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:28.751420 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca8702-6a9f-49d3-9ef2-6ee34a5b6169 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:28.764405 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 167/339] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:28.766044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca8702-6a9f-49d3-9ef2-6ee34a5b6169 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:28.803847 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:28.845087 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 173/340] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:28.875152 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca8702-6a9f-49d3-9ef2-6ee34a5b6169 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:28.883405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfd399-8be0-4f4b-8dc2-194f5eef5fa4 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:28.910930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fe4c3-6eea-4306-a9d4-0cbbd30251af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:28.911586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fe4c3-6eea-4306-a9d4-0cbbd30251af None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.911870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fe4c3-6eea-4306-a9d4-0cbbd30251af None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.912151 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fe4c3-6eea-4306-a9d4-0cbbd30251af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.929071 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:28.995377 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fe4c3-6eea-4306-a9d4-0cbbd30251af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:29.001505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fe4c3-6eea-4306-a9d4-0cbbd30251af None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:29.055821 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfd399-8be0-4f4b-8dc2-194f5eef5fa4 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:29.085203 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:29.134355 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ce8bc-f0bc-47ca-bfb2-d7d1c155cf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:29.135174 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ce8bc-f0bc-47ca-bfb2-d7d1c155cf70 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:29.135451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ce8bc-f0bc-47ca-bfb2-d7d1c155cf70 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:29.135720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ce8bc-f0bc-47ca-bfb2-d7d1c155cf70 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:29.160033 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ce8bc-f0bc-47ca-bfb2-d7d1c155cf70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:29.170870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-f10ce8bc-f0bc-47ca-bfb2-d7d1c155cf70 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:29.170870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:29.170870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:29.170870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:29.170870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:29.170870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:29.170870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:29.170870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:29.170870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:29.170870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:29.170870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:29.170870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:29.170870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:29.170870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:29.170870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:29.170870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:29.171752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:29.171752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:29.171752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:29.171752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:29.171752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:29.171752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:29.171752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:29.171752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:29.171752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:29.171752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:29.171752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:29.171752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:29.171752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:29.171752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:29.171752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:29.171752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:29.172592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:29.172592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:29.172592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:29.172592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:29.172592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:29.172592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:29.172592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:29.172592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:29.172592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:29.172592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:29.172592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:29.172592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:29.173176 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 174/341] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:29.185117 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ce8bc-f0bc-47ca-bfb2-d7d1c155cf70 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:29.212276 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 168/342] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:29.218239 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fe4c3-6eea-4306-a9d4-0cbbd30251af None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:29.341426 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:29.391096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ce8bc-f0bc-47ca-bfb2-d7d1c155cf70 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:29.475154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de5707-b7b5-4a1a-9097-e034f639a8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:29.475896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de5707-b7b5-4a1a-9097-e034f639a8aa None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:29.476184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de5707-b7b5-4a1a-9097-e034f639a8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:29.476491 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de5707-b7b5-4a1a-9097-e034f639a8aa None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:29.477319 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de5707-b7b5-4a1a-9097-e034f639a8aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:29.477639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de5707-b7b5-4a1a-9097-e034f639a8aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:29.479008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de5707-b7b5-4a1a-9097-e034f639a8aa None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:29.489364 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 175/343] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:29.494769 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de5707-b7b5-4a1a-9097-e034f639a8aa None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:29.514585 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fe4c3-6eea-4306-a9d4-0cbbd30251af None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:29.586043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474abbe-f6e2-4ee0-afef-613bae4536cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:29.586920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474abbe-f6e2-4ee0-afef-613bae4536cc None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:29.587207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474abbe-f6e2-4ee0-afef-613bae4536cc None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:29.587487 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474abbe-f6e2-4ee0-afef-613bae4536cc None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:29.588742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474abbe-f6e2-4ee0-afef-613bae4536cc None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:29.590233 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474abbe-f6e2-4ee0-afef-613bae4536cc None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:29.592029 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 169/344] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:29 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:29.597511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474abbe-f6e2-4ee0-afef-613bae4536cc None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:29.603864 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:29.661335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de5707-b7b5-4a1a-9097-e034f639a8aa None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:29.719572 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebbdf6-e858-4b5a-97cb-870182265327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:29.720555 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebbdf6-e858-4b5a-97cb-870182265327 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:29.720979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebbdf6-e858-4b5a-97cb-870182265327 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:29.721393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebbdf6-e858-4b5a-97cb-870182265327 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:29.742061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474abbe-f6e2-4ee0-afef-613bae4536cc None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:29.753491 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:29.789993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c22dbf-9c00-4bdb-9c3f-337fc0296a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:29.790869 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c22dbf-9c00-4bdb-9c3f-337fc0296a50 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:29.791168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c22dbf-9c00-4bdb-9c3f-337fc0296a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:29.791447 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c22dbf-9c00-4bdb-9c3f-337fc0296a50 None admin] PATH_INFO: `/v3/groups/0719110570b54fdf93607daf15940577` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:29.802524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c22dbf-9c00-4bdb-9c3f-337fc0296a50 None admin] RBAC: Authorizing `identity:get_group(group_id=0719110570b54fdf93607daf15940577)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:29.804559 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c22dbf-9c00-4bdb-9c3f-337fc0296a50 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:29.806604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 170/345] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:29 2022] GET /identity/v3/groups/0719110570b54fdf93607daf15940577 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:29.812428 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c22dbf-9c00-4bdb-9c3f-337fc0296a50 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:29.836791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebbdf6-e858-4b5a-97cb-870182265327 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:29.838394 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebbdf6-e858-4b5a-97cb-870182265327 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:29.896814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:29.951424 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c22dbf-9c00-4bdb-9c3f-337fc0296a50 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:29.999768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91af94-3581-4f55-a4e8-effa1c017fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:30.000467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91af94-3581-4f55-a4e8-effa1c017fae None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.000757 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91af94-3581-4f55-a4e8-effa1c017fae None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.001040 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91af94-3581-4f55-a4e8-effa1c017fae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.001892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91af94-3581-4f55-a4e8-effa1c017fae None admin] RBAC: Adding query filter params (group.id=0719110570b54fdf93607daf15940577, role.id=18f0dc41b78c4b52b6912225928ea388, scope.project.id=711e8b07480d478a80cbefc6fe30467f) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:30.002214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91af94-3581-4f55-a4e8-effa1c017fae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:30.003674 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91af94-3581-4f55-a4e8-effa1c017fae None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:30.009984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 171/346] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:29 2022] GET /identity/v3/role_assignments?group.id=0719110570b54fdf93607daf15940577&scope.project.id=711e8b07480d478a80cbefc6fe30467f&role.id=18f0dc41b78c4b52b6912225928ea388 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:30.049118 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 176/347] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:10:30.055452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91af94-3581-4f55-a4e8-effa1c017fae None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:30.187725 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91af94-3581-4f55-a4e8-effa1c017fae None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:30.224020 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:30.229635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5350f-094c-4e5d-b4e8-44d3c0ff8efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:30.230515 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5350f-094c-4e5d-b4e8-44d3c0ff8efa None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.230877 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5350f-094c-4e5d-b4e8-44d3c0ff8efa None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.231189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5350f-094c-4e5d-b4e8-44d3c0ff8efa None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.232500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5350f-094c-4e5d-b4e8-44d3c0ff8efa None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:30.234465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5350f-094c-4e5d-b4e8-44d3c0ff8efa None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:30.237738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 172/348] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:30 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:30.243994 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebbdf6-e858-4b5a-97cb-870182265327 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:30.248174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df704945-6778-4b02-8a4e-95d281c5d5a9 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.248488 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df704945-6778-4b02-8a4e-95d281c5d5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.248780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df704945-6778-4b02-8a4e-95d281c5d5a9 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.250413 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 173/349] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:30.261921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e4a8b-196f-48da-84b2-428d06e96d17 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.262222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e4a8b-196f-48da-84b2-428d06e96d17 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.262489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e4a8b-196f-48da-84b2-428d06e96d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.286205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-423e4a8b-196f-48da-84b2-428d06e96d17 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:30.289150 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-423e4a8b-196f-48da-84b2-428d06e96d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:30.320430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:30.391880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebbdf6-e858-4b5a-97cb-870182265327 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:30.414497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 174/350] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:30.416075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c800be-fecb-4dc8-a8dc-0909caa25190 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.416502 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c800be-fecb-4dc8-a8dc-0909caa25190 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.417028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c800be-fecb-4dc8-a8dc-0909caa25190 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.418527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 175/351] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:30.419792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734e9ca-e785-4887-adce-65ba24b30865 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.424340 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734e9ca-e785-4887-adce-65ba24b30865 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.424972 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734e9ca-e785-4887-adce-65ba24b30865 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.426887 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 176/352] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:30.433006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0a77e-e890-4051-922e-835474b226d7 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.433565 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0a77e-e890-4051-922e-835474b226d7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.434298 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0a77e-e890-4051-922e-835474b226d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.438913 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d18e4-45ba-430d-87a3-05e6e9d45a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:30.439725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d18e4-45ba-430d-87a3-05e6e9d45a52 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.440010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d18e4-45ba-430d-87a3-05e6e9d45a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.440299 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d18e4-45ba-430d-87a3-05e6e9d45a52 None admin] PATH_INFO: `/v3/groups/0719110570b54fdf93607daf15940577` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.445116 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d18e4-45ba-430d-87a3-05e6e9d45a52 None admin] RBAC: Authorizing `identity:get_group(group_id=0719110570b54fdf93607daf15940577)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:30.446567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d18e4-45ba-430d-87a3-05e6e9d45a52 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:30.448776 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 177/353] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:30 2022] GET /identity/v3/groups/0719110570b54fdf93607daf15940577 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:30.449981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bd395-acd3-45ee-b397-ad58a9c861d1 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.450273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bd395-acd3-45ee-b397-ad58a9c861d1 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.450548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bd395-acd3-45ee-b397-ad58a9c861d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.456376 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3bc0a77e-e890-4051-922e-835474b226d7 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:30.457809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3bc0a77e-e890-4051-922e-835474b226d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:30.474044 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-1e3bd395-acd3-45ee-b397-ad58a9c861d1 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:30.475274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-1e3bd395-acd3-45ee-b397-ad58a9c861d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:30.540456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 177/354] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:30.541630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0a77e-e890-4051-922e-835474b226d7 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:30.555278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 178/355] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:30.561297 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bd395-acd3-45ee-b397-ad58a9c861d1 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:30.604667 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:30.645616 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:30.647137 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0a77e-e890-4051-922e-835474b226d7 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:30.691438 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3353e-254f-4303-a1b6-a69d912c5be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:30.692375 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3353e-254f-4303-a1b6-a69d912c5be5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.693142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3353e-254f-4303-a1b6-a69d912c5be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.693573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3353e-254f-4303-a1b6-a69d912c5be5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.694613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3353e-254f-4303-a1b6-a69d912c5be5 None admin] RBAC: Adding query filter params (group.id=0719110570b54fdf93607daf15940577, role.id=abe200f23fc84ba397dd8c67874f0f2a, scope.project.id=711e8b07480d478a80cbefc6fe30467f) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:30.695075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3353e-254f-4303-a1b6-a69d912c5be5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:30.697248 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3353e-254f-4303-a1b6-a69d912c5be5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:30.704744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 178/356] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:30 2022] GET /identity/v3/role_assignments?group.id=0719110570b54fdf93607daf15940577&scope.project.id=711e8b07480d478a80cbefc6fe30467f&role.id=abe200f23fc84ba397dd8c67874f0f2a => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:30.705888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3353e-254f-4303-a1b6-a69d912c5be5 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:30.713843 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bd395-acd3-45ee-b397-ad58a9c861d1 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:30.758551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486ee9f-7af1-49bc-bc85-04301176e06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:30.759520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486ee9f-7af1-49bc-bc85-04301176e06b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.759920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486ee9f-7af1-49bc-bc85-04301176e06b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.760323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486ee9f-7af1-49bc-bc85-04301176e06b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.774570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486ee9f-7af1-49bc-bc85-04301176e06b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:30.783837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-d486ee9f-7af1-49bc-bc85-04301176e06b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:30.783837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:30.783837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:30.783837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:30.783837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:30.783837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:30.783837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:30.783837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:30.783837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:30.783837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:30.783837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:30.783837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:30.783837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:30.783837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:30.783837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:30.783837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:30.785040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:30.785040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:30.785040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:30.785040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:30.785040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:30.785040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:30.785040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:30.785040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:30.785040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:30.785040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:30.785040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:30.785040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:30.785040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:30.785040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:30.785040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:30.785040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:30.785850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:30.785850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:30.785850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:30.785850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:30.785850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:30.785850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:30.785850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:30.785850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:30.785850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:30.785850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:30.785850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:30.785850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:30.786472 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 179/357] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:30.787979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486ee9f-7af1-49bc-bc85-04301176e06b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:30.833525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3353e-254f-4303-a1b6-a69d912c5be5 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:30.860172 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:30.874468 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a16aa5-755a-4afd-b46b-30bbfb1b685a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:30.875601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a16aa5-755a-4afd-b46b-30bbfb1b685a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.876065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a16aa5-755a-4afd-b46b-30bbfb1b685a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.876580 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a16aa5-755a-4afd-b46b-30bbfb1b685a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.888360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a16aa5-755a-4afd-b46b-30bbfb1b685a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:30.897311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-47a16aa5-755a-4afd-b46b-30bbfb1b685a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:30.897311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:30.897311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:30.897311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:30.897311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:30.897311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:30.897311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:30.897311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:30.897311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:30.897311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:30.897311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:30.897311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:30.897311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:30.897311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:30.897311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:30.897311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:30.898277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:30.898277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:30.898277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:30.898277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:30.898277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:30.898277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:30.898277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:30.898277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:30.898277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:30.898277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:30.898277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:30.898277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:30.898277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:30.898277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:30.898277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:30.898277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:30.899052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:30.899052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:30.899052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:30.899052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:30.899052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:30.899052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:30.899052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:30.899052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:30.899052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:30.899052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:30.899052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:30.899052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:30.901573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 179/358] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:30.902561 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a16aa5-755a-4afd-b46b-30bbfb1b685a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:30.916278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486ee9f-7af1-49bc-bc85-04301176e06b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:30.924675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:30.956206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33cafb-0036-4cad-aac6-dfca0987222e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:30.957376 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33cafb-0036-4cad-aac6-dfca0987222e None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.957896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33cafb-0036-4cad-aac6-dfca0987222e None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.958367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33cafb-0036-4cad-aac6-dfca0987222e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.971528 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d33cafb-0036-4cad-aac6-dfca0987222e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:30.980430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-5d33cafb-0036-4cad-aac6-dfca0987222e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:30.980430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:30.980430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:30.980430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:30.980430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:30.980430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:30.980430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:30.980430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:30.980430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:30.980430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:30.980430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:30.980430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:30.980430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:30.980430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:30.980430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:30.980430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:30.981373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:30.981373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:30.981373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:30.981373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:30.981373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:30.981373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:30.981373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:30.981373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:30.981373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:30.981373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:30.981373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:30.981373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:30.981373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:30.981373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:30.981373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:30.981373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:30.982607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:30.982607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:30.982607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:30.982607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:30.982607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:30.982607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:30.982607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:30.982607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:30.982607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:30.982607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:30.982607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:30.982607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:30.984808 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 180/359] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:30.985947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33cafb-0036-4cad-aac6-dfca0987222e None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:31.055474 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a16aa5-755a-4afd-b46b-30bbfb1b685a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:31.103711 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33cafb-0036-4cad-aac6-dfca0987222e None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:31.104511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f60468-82d4-4cbe-b9df-6ee174303945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:31.105446 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f60468-82d4-4cbe-b9df-6ee174303945 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.105736 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f60468-82d4-4cbe-b9df-6ee174303945 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.106015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f60468-82d4-4cbe-b9df-6ee174303945 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.106781 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f60468-82d4-4cbe-b9df-6ee174303945 None admin] RBAC: Adding query filter params (name=member) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:31.108025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f60468-82d4-4cbe-b9df-6ee174303945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:31.109515 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f60468-82d4-4cbe-b9df-6ee174303945 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:31.120400 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 180/360] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:31.121739 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d7e8f-1dea-4172-b610-3bd8387d3612 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.122030 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d7e8f-1dea-4172-b610-3bd8387d3612 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.122309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d7e8f-1dea-4172-b610-3bd8387d3612 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.123671 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 181/361] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:31.126397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d7e8f-1dea-4172-b610-3bd8387d3612 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:31.151703 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2672b7-4fb2-4f98-8624-01e3f39605ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:31.152777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2672b7-4fb2-4f98-8624-01e3f39605ca None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.153346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2672b7-4fb2-4f98-8624-01e3f39605ca None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.153825 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2672b7-4fb2-4f98-8624-01e3f39605ca None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.154828 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2672b7-4fb2-4f98-8624-01e3f39605ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:31.155338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2672b7-4fb2-4f98-8624-01e3f39605ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:31.156979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2672b7-4fb2-4f98-8624-01e3f39605ca None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:31.166840 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 181/362] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:31.168099 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2672b7-4fb2-4f98-8624-01e3f39605ca None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:31.186015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:31.198829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:31.248963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d7e8f-1dea-4172-b610-3bd8387d3612 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:31.278796 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2672b7-4fb2-4f98-8624-01e3f39605ca None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:31.289653 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee7ca9-3373-43e0-8161-265478bd3bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:31.290666 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee7ca9-3373-43e0-8161-265478bd3bae None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.291133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee7ca9-3373-43e0-8161-265478bd3bae None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.291583 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee7ca9-3373-43e0-8161-265478bd3bae None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.292661 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee7ca9-3373-43e0-8161-265478bd3bae None admin] RBAC: Adding query filter params (name=reader) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:31.293163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee7ca9-3373-43e0-8161-265478bd3bae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:31.294702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee7ca9-3373-43e0-8161-265478bd3bae None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:31.305556 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 182/363] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:31.311715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d2ff1-193e-49a2-a786-629bb9379e15 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.312256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d2ff1-193e-49a2-a786-629bb9379e15 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.312806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d2ff1-193e-49a2-a786-629bb9379e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.317772 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f52dc-f89b-43af-8496-4f8e4f5e915f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:31.318573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f52dc-f89b-43af-8496-4f8e4f5e915f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.318973 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f52dc-f89b-43af-8496-4f8e4f5e915f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.319325 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f52dc-f89b-43af-8496-4f8e4f5e915f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.335842 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-206d2ff1-193e-49a2-a786-629bb9379e15 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:31.337414 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-206d2ff1-193e-49a2-a786-629bb9379e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:31.418079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f52dc-f89b-43af-8496-4f8e4f5e915f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:31.420096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f52dc-f89b-43af-8496-4f8e4f5e915f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:31.437867 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 183/364] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:31.439377 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d2ff1-193e-49a2-a786-629bb9379e15 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:31.460321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:31.538874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:31.545809 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 182/365] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:31.546691 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f52dc-f89b-43af-8496-4f8e4f5e915f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:31.627788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d2ff1-193e-49a2-a786-629bb9379e15 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:31.669189 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f52dc-f89b-43af-8496-4f8e4f5e915f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:31.676342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1355fe8-af98-46dc-a1b0-dfc3fc4ca33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:31.677009 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1355fe8-af98-46dc-a1b0-dfc3fc4ca33f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.677297 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1355fe8-af98-46dc-a1b0-dfc3fc4ca33f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.677578 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1355fe8-af98-46dc-a1b0-dfc3fc4ca33f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.708859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:31.718899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac568cf-88c7-4e60-a95a-039fb6f81d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:31.720114 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac568cf-88c7-4e60-a95a-039fb6f81d9e None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.720627 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac568cf-88c7-4e60-a95a-039fb6f81d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.721168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac568cf-88c7-4e60-a95a-039fb6f81d9e None admin] PATH_INFO: `/v3/users/d5fa4d4a59de4ab9b57d76bb7acd4278` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.722562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac568cf-88c7-4e60-a95a-039fb6f81d9e None admin] RBAC: Authorizing `identity:get_user(user_id=d5fa4d4a59de4ab9b57d76bb7acd4278)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:31.724181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac568cf-88c7-4e60-a95a-039fb6f81d9e None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:31.726525 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 183/366] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:31 2022] GET /identity/v3/users/d5fa4d4a59de4ab9b57d76bb7acd4278 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:31.727461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac568cf-88c7-4e60-a95a-039fb6f81d9e None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:31.768475 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1355fe8-af98-46dc-a1b0-dfc3fc4ca33f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:31.771262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1355fe8-af98-46dc-a1b0-dfc3fc4ca33f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:31.820044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:31.865281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac568cf-88c7-4e60-a95a-039fb6f81d9e None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:31.891981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 184/367] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:31.898123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1355fe8-af98-46dc-a1b0-dfc3fc4ca33f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:31.907927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70f4a6-c67a-4765-b47c-3b18fe21f75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:31.908773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70f4a6-c67a-4765-b47c-3b18fe21f75f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.909064 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70f4a6-c67a-4765-b47c-3b18fe21f75f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.909346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70f4a6-c67a-4765-b47c-3b18fe21f75f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.924470 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb70f4a6-c67a-4765-b47c-3b18fe21f75f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:31.935685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-cb70f4a6-c67a-4765-b47c-3b18fe21f75f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:31.935685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:31.935685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:31.935685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:31.935685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:31.935685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:31.935685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:31.935685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:31.935685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:31.935685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:31.935685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:31.935685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:31.935685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:31.935685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:31.935685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:31.935685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:31.936662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:31.936662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:31.936662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:31.936662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:31.936662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:31.936662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:31.936662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:31.936662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:31.936662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:31.936662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:31.936662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:31.936662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:31.936662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:31.936662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:31.936662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:31.936662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:31.937493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:31.937493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:31.937493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:31.937493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:31.937493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:31.937493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:31.937493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:31.937493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:31.937493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:31.937493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:31.937493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:31.937493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:31.938128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 184/368] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:31.938700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70f4a6-c67a-4765-b47c-3b18fe21f75f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.016793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:32.034349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1355fe8-af98-46dc-a1b0-dfc3fc4ca33f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.076581 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be198d-a7ac-45c7-a5d2-14c4ce286521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.076773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70f4a6-c67a-4765-b47c-3b18fe21f75f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.077384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be198d-a7ac-45c7-a5d2-14c4ce286521 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.077667 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be198d-a7ac-45c7-a5d2-14c4ce286521 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.078052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be198d-a7ac-45c7-a5d2-14c4ce286521 None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.079413 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be198d-a7ac-45c7-a5d2-14c4ce286521 None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:32.080909 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be198d-a7ac-45c7-a5d2-14c4ce286521 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.083033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 185/369] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:31 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:32.084060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be198d-a7ac-45c7-a5d2-14c4ce286521 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.094018 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:32.122961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2e803-8fcc-49a1-9e4e-23ddc34b2d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.123787 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2e803-8fcc-49a1-9e4e-23ddc34b2d30 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.124072 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2e803-8fcc-49a1-9e4e-23ddc34b2d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.124689 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2e803-8fcc-49a1-9e4e-23ddc34b2d30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.125833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2e803-8fcc-49a1-9e4e-23ddc34b2d30 None admin] RBAC: Adding query filter params (role.id=ba4641dd50c64f80a618b95aaaa321cb, user.id=d5fa4d4a59de4ab9b57d76bb7acd4278, scope.system=all) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:32.126154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2e803-8fcc-49a1-9e4e-23ddc34b2d30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:32.127781 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2e803-8fcc-49a1-9e4e-23ddc34b2d30 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.148381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 185/370] 10.222.0.43 () {56 vars in 1308 bytes} [Wed Jul 20 19:10:31 2022] GET /identity/v3/role_assignments?user.id=d5fa4d4a59de4ab9b57d76bb7acd4278&scope.system=all&role.id=ba4641dd50c64f80a618b95aaaa321cb => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:32.149362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2e803-8fcc-49a1-9e4e-23ddc34b2d30 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.251118 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be198d-a7ac-45c7-a5d2-14c4ce286521 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.326529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9d0a2-7275-499d-b211-02f3e81e1d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.327679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9d0a2-7275-499d-b211-02f3e81e1d31 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.328872 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9d0a2-7275-499d-b211-02f3e81e1d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.329172 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9d0a2-7275-499d-b211-02f3e81e1d31 None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.330646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9d0a2-7275-499d-b211-02f3e81e1d31 None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:32.332131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9d0a2-7275-499d-b211-02f3e81e1d31 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.337689 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 186/371] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:32 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:32.342790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9d0a2-7275-499d-b211-02f3e81e1d31 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.366847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2e803-8fcc-49a1-9e4e-23ddc34b2d30 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.386681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:32.418242 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d760f7-8d23-4ac4-8d64-9cd7045ae1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.418932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d760f7-8d23-4ac4-8d64-9cd7045ae1dd None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.419225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d760f7-8d23-4ac4-8d64-9cd7045ae1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.419510 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d760f7-8d23-4ac4-8d64-9cd7045ae1dd None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.420274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d760f7-8d23-4ac4-8d64-9cd7045ae1dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:32.420610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d760f7-8d23-4ac4-8d64-9cd7045ae1dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:32.422680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d760f7-8d23-4ac4-8d64-9cd7045ae1dd None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.434422 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 186/372] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:32.435476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d760f7-8d23-4ac4-8d64-9cd7045ae1dd None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.466491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:32.501381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9d0a2-7275-499d-b211-02f3e81e1d31 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.560252 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b548ec-4750-454d-8bab-6771866d3173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.562109 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b548ec-4750-454d-8bab-6771866d3173 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.562571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b548ec-4750-454d-8bab-6771866d3173 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.563023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b548ec-4750-454d-8bab-6771866d3173 None admin] PATH_INFO: `/v3/groups/0719110570b54fdf93607daf15940577` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.565072 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b548ec-4750-454d-8bab-6771866d3173 None admin] RBAC: Authorizing `identity:get_group(group_id=0719110570b54fdf93607daf15940577)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:32.567273 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b548ec-4750-454d-8bab-6771866d3173 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.570539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 187/373] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:32 2022] GET /identity/v3/groups/0719110570b54fdf93607daf15940577 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:32.571740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b548ec-4750-454d-8bab-6771866d3173 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.588423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d760f7-8d23-4ac4-8d64-9cd7045ae1dd None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.645199 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0911c89-7d91-45e3-8044-4c1af7c8ffca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.646607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0911c89-7d91-45e3-8044-4c1af7c8ffca None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.647650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0911c89-7d91-45e3-8044-4c1af7c8ffca None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.647941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0911c89-7d91-45e3-8044-4c1af7c8ffca None admin] PATH_INFO: `/v3/groups/0719110570b54fdf93607daf15940577` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.651033 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0911c89-7d91-45e3-8044-4c1af7c8ffca None admin] RBAC: Authorizing `identity:get_group(group_id=0719110570b54fdf93607daf15940577)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:32.652686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0911c89-7d91-45e3-8044-4c1af7c8ffca None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.654533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 187/374] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:32 2022] GET /identity/v3/groups/0719110570b54fdf93607daf15940577 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:32.657364 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0911c89-7d91-45e3-8044-4c1af7c8ffca None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.696950 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b548ec-4750-454d-8bab-6771866d3173 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.708596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:32.739538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf9908-7f3b-4e2c-8411-91a1316845a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.741060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf9908-7f3b-4e2c-8411-91a1316845a2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.741354 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf9908-7f3b-4e2c-8411-91a1316845a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.741636 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf9908-7f3b-4e2c-8411-91a1316845a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.758553 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:32.786032 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0911c89-7d91-45e3-8044-4c1af7c8ffca None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.825649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf9908-7f3b-4e2c-8411-91a1316845a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:32.827242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf9908-7f3b-4e2c-8411-91a1316845a2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.834920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb76695-ca65-4c84-9f71-087d49159b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.836486 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb76695-ca65-4c84-9f71-087d49159b46 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.836486 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb76695-ca65-4c84-9f71-087d49159b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.836486 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb76695-ca65-4c84-9f71-087d49159b46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.837837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb76695-ca65-4c84-9f71-087d49159b46 None admin] RBAC: Adding query filter params (group.id=0719110570b54fdf93607daf15940577, role.id=18f0dc41b78c4b52b6912225928ea388, scope.project.id=359a065a7d2c49fea870a82587a5d51e) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:32.837837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb76695-ca65-4c84-9f71-087d49159b46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:32.839351 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb76695-ca65-4c84-9f71-087d49159b46 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.846744 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 188/375] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:32 2022] GET /identity/v3/role_assignments?group.id=0719110570b54fdf93607daf15940577&scope.project.id=359a065a7d2c49fea870a82587a5d51e&role.id=18f0dc41b78c4b52b6912225928ea388 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:32.848732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08eab2-82e0-40ca-b94c-c66d4b86a764 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.849297 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08eab2-82e0-40ca-b94c-c66d4b86a764 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.849820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08eab2-82e0-40ca-b94c-c66d4b86a764 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.851386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 189/376] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:32.851957 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08eab2-82e0-40ca-b94c-c66d4b86a764 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.994507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 188/377] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:32.996282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58921f2f-e85c-4ee6-96db-608070d74c9d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.996802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58921f2f-e85c-4ee6-96db-608070d74c9d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.997320 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58921f2f-e85c-4ee6-96db-608070d74c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.002618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08eab2-82e0-40ca-b94c-c66d4b86a764 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.014509 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:33.021748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-58921f2f-e85c-4ee6-96db-608070d74c9d None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:33.023334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-58921f2f-e85c-4ee6-96db-608070d74c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:33.048406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626ad27-e464-47f0-85f0-165144a31ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.049079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626ad27-e464-47f0-85f0-165144a31ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.049423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626ad27-e464-47f0-85f0-165144a31ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.049701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626ad27-e464-47f0-85f0-165144a31ec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.050544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626ad27-e464-47f0-85f0-165144a31ec6 None admin] RBAC: Adding query filter params (group.id=0719110570b54fdf93607daf15940577, role.id=abe200f23fc84ba397dd8c67874f0f2a, scope.project.id=359a065a7d2c49fea870a82587a5d51e) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:33.050854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626ad27-e464-47f0-85f0-165144a31ec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:33.052320 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626ad27-e464-47f0-85f0-165144a31ec6 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:33.060135 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 190/378] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:32 2022] GET /identity/v3/role_assignments?group.id=0719110570b54fdf93607daf15940577&scope.project.id=359a065a7d2c49fea870a82587a5d51e&role.id=abe200f23fc84ba397dd8c67874f0f2a => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:33.060982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626ad27-e464-47f0-85f0-165144a31ec6 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.081988 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:33.117898 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 189/379] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:33.140063 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58921f2f-e85c-4ee6-96db-608070d74c9d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.193991 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626ad27-e464-47f0-85f0-165144a31ec6 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.241091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bb9ae-1adc-4160-9fcc-5aa22f07dfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.241994 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bb9ae-1adc-4160-9fcc-5aa22f07dfb4 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.242273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bb9ae-1adc-4160-9fcc-5aa22f07dfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.242546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bb9ae-1adc-4160-9fcc-5aa22f07dfb4 None admin] PATH_INFO: `/v3/projects/39f878600fda47389da52621d6e0947c` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.246932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bb9ae-1adc-4160-9fcc-5aa22f07dfb4 None admin] RBAC: Authorizing `identity:get_project(project_id=39f878600fda47389da52621d6e0947c)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:33.249696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bb9ae-1adc-4160-9fcc-5aa22f07dfb4 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:33.251585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 191/380] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/projects/39f878600fda47389da52621d6e0947c => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:33.257936 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bb9ae-1adc-4160-9fcc-5aa22f07dfb4 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.325443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:33.340211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58921f2f-e85c-4ee6-96db-608070d74c9d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.392508 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec36511-12f1-403e-ab8e-8718dd9a2479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.393789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec36511-12f1-403e-ab8e-8718dd9a2479 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.394352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec36511-12f1-403e-ab8e-8718dd9a2479 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.394870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec36511-12f1-403e-ab8e-8718dd9a2479 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.400486 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bb9ae-1adc-4160-9fcc-5aa22f07dfb4 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.408062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec36511-12f1-403e-ab8e-8718dd9a2479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:33.416501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-1ec36511-12f1-403e-ab8e-8718dd9a2479 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:33.416501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:33.416501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:33.416501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:33.416501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:33.416501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:33.416501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:33.416501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:33.416501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:33.416501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:33.416501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:33.416501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:33.416501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:33.416501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:33.416501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:33.416501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:33.417435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:33.417435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:33.417435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:33.417435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:33.417435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:33.417435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:33.417435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:33.417435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:33.417435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:33.417435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:33.417435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:33.417435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:33.417435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:33.417435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:33.417435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:33.417435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:33.418397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:33.418397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:33.418397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:33.418397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:33.418397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:33.418397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:33.418397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:33.418397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:33.418397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:33.418397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:33.418397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:33.418397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:33.424021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 190/381] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:33.427275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c18eb9-2cb6-4132-b439-4879a1d6f804 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.428248 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c18eb9-2cb6-4132-b439-4879a1d6f804 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.429084 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c18eb9-2cb6-4132-b439-4879a1d6f804 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.436417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 191/382] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:33.438059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c18eb9-2cb6-4132-b439-4879a1d6f804 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.445518 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e033cc-8f00-4fcd-995f-48c69da286ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.446395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e033cc-8f00-4fcd-995f-48c69da286ab None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.446678 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e033cc-8f00-4fcd-995f-48c69da286ab None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.446951 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e033cc-8f00-4fcd-995f-48c69da286ab None admin] PATH_INFO: `/v3/groups/3bde401d70ac4c90a2c020d6c7c337a9` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.453456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e033cc-8f00-4fcd-995f-48c69da286ab None admin] RBAC: Authorizing `identity:get_group(group_id=3bde401d70ac4c90a2c020d6c7c337a9)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:33.455004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e033cc-8f00-4fcd-995f-48c69da286ab None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:33.456986 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 192/383] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/groups/3bde401d70ac4c90a2c020d6c7c337a9 => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:33.458209 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3a614-a709-42ff-8b75-dd178537f71a None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.458499 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3a614-a709-42ff-8b75-dd178537f71a None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.458779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3a614-a709-42ff-8b75-dd178537f71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.480778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-f1b3a614-a709-42ff-8b75-dd178537f71a None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:33.481999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f1b3a614-a709-42ff-8b75-dd178537f71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:33.489044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:33.569413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 193/384] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:33.570398 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c18eb9-2cb6-4132-b439-4879a1d6f804 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.572196 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3a614-a709-42ff-8b75-dd178537f71a None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.609203 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:33.609693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0ac9d-6a54-452b-b43b-ec397acdb2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.610920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0ac9d-6a54-452b-b43b-ec397acdb2be None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.611454 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0ac9d-6a54-452b-b43b-ec397acdb2be None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.611954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0ac9d-6a54-452b-b43b-ec397acdb2be None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.612955 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0ac9d-6a54-452b-b43b-ec397acdb2be None admin] RBAC: Adding query filter params (name=member) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:33.613485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0ac9d-6a54-452b-b43b-ec397acdb2be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:33.615138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0ac9d-6a54-452b-b43b-ec397acdb2be None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:33.626446 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 192/385] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:33.627596 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0ac9d-6a54-452b-b43b-ec397acdb2be None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.688246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3a614-a709-42ff-8b75-dd178537f71a None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.726588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedac400-fd88-4919-87ec-cf02da9a9739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.727252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedac400-fd88-4919-87ec-cf02da9a9739 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.727532 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedac400-fd88-4919-87ec-cf02da9a9739 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.727804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedac400-fd88-4919-87ec-cf02da9a9739 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.728671 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedac400-fd88-4919-87ec-cf02da9a9739 None admin] RBAC: Adding query filter params (group.id=3bde401d70ac4c90a2c020d6c7c337a9, role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=39f878600fda47389da52621d6e0947c) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:33.729003 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedac400-fd88-4919-87ec-cf02da9a9739 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:33.730555 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedac400-fd88-4919-87ec-cf02da9a9739 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:33.733851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:33.737694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 194/386] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/role_assignments?group.id=3bde401d70ac4c90a2c020d6c7c337a9&scope.project.id=39f878600fda47389da52621d6e0947c&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:33.738618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedac400-fd88-4919-87ec-cf02da9a9739 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.742093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0ac9d-6a54-452b-b43b-ec397acdb2be None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.784197 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b254d55-3b35-4d58-922a-b440537630f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.785555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b254d55-3b35-4d58-922a-b440537630f7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.786051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b254d55-3b35-4d58-922a-b440537630f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.786529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b254d55-3b35-4d58-922a-b440537630f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.799753 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b254d55-3b35-4d58-922a-b440537630f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:33.809373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-9b254d55-3b35-4d58-922a-b440537630f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:33.809373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:33.809373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:33.809373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:33.809373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:33.809373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:33.809373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:33.809373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:33.809373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:33.809373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:33.809373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:33.809373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:33.809373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:33.809373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:33.809373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:33.809373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:33.810313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:33.810313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:33.810313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:33.810313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:33.810313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:33.810313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:33.810313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:33.810313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:33.810313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:33.810313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:33.810313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:33.810313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:33.810313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:33.810313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:33.810313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:33.810313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:33.811107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:33.811107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:33.811107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:33.811107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:33.811107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:33.811107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:33.811107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:33.811107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:33.811107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:33.811107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:33.811107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:33.811107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:33.813855 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 193/387] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:33.821347 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b254d55-3b35-4d58-922a-b440537630f7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.848725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:33.856343 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedac400-fd88-4919-87ec-cf02da9a9739 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.892751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36771b22-bf06-4e41-97cc-677c19b97be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.893628 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36771b22-bf06-4e41-97cc-677c19b97be0 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.893908 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36771b22-bf06-4e41-97cc-677c19b97be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.894182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36771b22-bf06-4e41-97cc-677c19b97be0 None admin] PATH_INFO: `/v3/groups/0719110570b54fdf93607daf15940577` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.896075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36771b22-bf06-4e41-97cc-677c19b97be0 None admin] RBAC: Authorizing `identity:get_group(group_id=0719110570b54fdf93607daf15940577)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:33.897827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36771b22-bf06-4e41-97cc-677c19b97be0 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:33.903773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 195/388] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/groups/0719110570b54fdf93607daf15940577 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:33.905312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36771b22-bf06-4e41-97cc-677c19b97be0 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.974064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b254d55-3b35-4d58-922a-b440537630f7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:34.013588 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266dcbd0-61f1-492c-b9d8-17ddf20ee092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:34.014622 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dcbd0-61f1-492c-b9d8-17ddf20ee092 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:34.015288 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dcbd0-61f1-492c-b9d8-17ddf20ee092 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:34.016067 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dcbd0-61f1-492c-b9d8-17ddf20ee092 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:34.017075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266dcbd0-61f1-492c-b9d8-17ddf20ee092 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:34.017571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266dcbd0-61f1-492c-b9d8-17ddf20ee092 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:34.019181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266dcbd0-61f1-492c-b9d8-17ddf20ee092 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:34.025388 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36771b22-bf06-4e41-97cc-677c19b97be0 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:34.031179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 194/389] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:34.036561 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266dcbd0-61f1-492c-b9d8-17ddf20ee092 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:34.060910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:34.104821 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5f6f8-162e-46ad-8dc0-363dcff5a29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:34.105568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5f6f8-162e-46ad-8dc0-363dcff5a29c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:34.105858 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5f6f8-162e-46ad-8dc0-363dcff5a29c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:34.106138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5f6f8-162e-46ad-8dc0-363dcff5a29c None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:34.109023 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5f6f8-162e-46ad-8dc0-363dcff5a29c None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:34.110778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5f6f8-162e-46ad-8dc0-363dcff5a29c None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:34.116497 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 196/390] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:34.119355 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5f6f8-162e-46ad-8dc0-363dcff5a29c None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:34.163530 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266dcbd0-61f1-492c-b9d8-17ddf20ee092 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:34.172701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:34.211890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440bf98-2339-4846-ba2d-ba6bdbdf47de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:34.213923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440bf98-2339-4846-ba2d-ba6bdbdf47de None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:34.214416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440bf98-2339-4846-ba2d-ba6bdbdf47de None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:34.214887 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440bf98-2339-4846-ba2d-ba6bdbdf47de None admin] PATH_INFO: `/v3/groups/0719110570b54fdf93607daf15940577` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:34.216390 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440bf98-2339-4846-ba2d-ba6bdbdf47de None admin] RBAC: Authorizing `identity:get_group(group_id=0719110570b54fdf93607daf15940577)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:34.218492 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440bf98-2339-4846-ba2d-ba6bdbdf47de None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:34.221131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 195/391] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:34 2022] GET /identity/v3/groups/0719110570b54fdf93607daf15940577 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:34.229353 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440bf98-2339-4846-ba2d-ba6bdbdf47de None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:34.252273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5f6f8-162e-46ad-8dc0-363dcff5a29c None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:34.311362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ab773-5a6c-43c0-8a84-dea4daca7d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:34.312269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ab773-5a6c-43c0-8a84-dea4daca7d72 None admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:34.312576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ab773-5a6c-43c0-8a84-dea4daca7d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:34.312854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ab773-5a6c-43c0-8a84-dea4daca7d72 None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f/groups/0719110570b54fdf93607daf15940577/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:34.316143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ab773-5a6c-43c0-8a84-dea4daca7d72 None admin] RBAC: Authorizing `identity:create_grant(project_id=711e8b07480d478a80cbefc6fe30467f, group_id=0719110570b54fdf93607daf15940577, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:34.320553 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ab773-5a6c-43c0-8a84-dea4daca7d72 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:34.349772 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 197/392] 10.222.0.43 () {58 vars in 1442 bytes} [Wed Jul 20 19:10:34 2022] PUT /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f/groups/0719110570b54fdf93607daf15940577/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:10:34.381952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:34.401531 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440bf98-2339-4846-ba2d-ba6bdbdf47de None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:34.443110 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e112ff-ac71-4bd8-9b71-7aea0eb7c557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:34.444064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e112ff-ac71-4bd8-9b71-7aea0eb7c557 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:34.444393 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e112ff-ac71-4bd8-9b71-7aea0eb7c557 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:34.444723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e112ff-ac71-4bd8-9b71-7aea0eb7c557 None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:34.445958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e112ff-ac71-4bd8-9b71-7aea0eb7c557 None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:34.447537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e112ff-ac71-4bd8-9b71-7aea0eb7c557 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:34.449660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 196/393] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:34 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:34.455967 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ab773-5a6c-43c0-8a84-dea4daca7d72 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:34.590538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:34.598444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ab773-5a6c-43c0-8a84-dea4daca7d72 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:34.640339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252e51b-f293-4860-9bab-669879932e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:34.641269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252e51b-f293-4860-9bab-669879932e31 None admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:34.641551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252e51b-f293-4860-9bab-669879932e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:34.641824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252e51b-f293-4860-9bab-669879932e31 None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f/groups/0719110570b54fdf93607daf15940577/roles/abe200f23fc84ba397dd8c67874f0f2a` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:34.644141 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252e51b-f293-4860-9bab-669879932e31 None admin] RBAC: Authorizing `identity:create_grant(project_id=711e8b07480d478a80cbefc6fe30467f, group_id=0719110570b54fdf93607daf15940577, role_id=abe200f23fc84ba397dd8c67874f0f2a)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:34.645762 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252e51b-f293-4860-9bab-669879932e31 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:34.675337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 198/394] 10.222.0.43 () {58 vars in 1442 bytes} [Wed Jul 20 19:10:34 2022] PUT /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f/groups/0719110570b54fdf93607daf15940577/roles/abe200f23fc84ba397dd8c67874f0f2a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:35.469055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc7188-0652-4ddb-a76a-8c69b860b751 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:35.469557 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc7188-0652-4ddb-a76a-8c69b860b751 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:35.470296 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc7188-0652-4ddb-a76a-8c69b860b751 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:35.471927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 197/395] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:35.477182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ceabf-9f04-4e9f-a1e0-d6b553489363 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:35.477616 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ceabf-9f04-4e9f-a1e0-d6b553489363 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:35.477787 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ceabf-9f04-4e9f-a1e0-d6b553489363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:35.498797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4f2ceabf-9f04-4e9f-a1e0-d6b553489363 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:35.499998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4f2ceabf-9f04-4e9f-a1e0-d6b553489363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:35.535414 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176d80f-411d-4e72-bf79-21821bde018f None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:35.536041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176d80f-411d-4e72-bf79-21821bde018f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:35.536572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176d80f-411d-4e72-bf79-21821bde018f None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:35.539540 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 198/396] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:35.544907 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d984729-cc03-4eca-9e7a-5414671df784 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:35.545630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d984729-cc03-4eca-9e7a-5414671df784 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:35.546127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d984729-cc03-4eca-9e7a-5414671df784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:35.564785 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4d984729-cc03-4eca-9e7a-5414671df784 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:35.566301 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4d984729-cc03-4eca-9e7a-5414671df784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:35.599253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 199/397] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:35.641982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ceabf-9f04-4e9f-a1e0-d6b553489363 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:35.664236 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 199/398] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:35.729681 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d984729-cc03-4eca-9e7a-5414671df784 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:35.763045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:35.792210 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ceabf-9f04-4e9f-a1e0-d6b553489363 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:35.861644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f17dee-19e6-4298-a3a5-f46b99c4e35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:35.863004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f17dee-19e6-4298-a3a5-f46b99c4e35c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:35.863639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f17dee-19e6-4298-a3a5-f46b99c4e35c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:35.864836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f17dee-19e6-4298-a3a5-f46b99c4e35c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:35.865988 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:35.888535 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f17dee-19e6-4298-a3a5-f46b99c4e35c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:35.904708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d984729-cc03-4eca-9e7a-5414671df784 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:35.907311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-f4f17dee-19e6-4298-a3a5-f46b99c4e35c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:35.907311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:35.907311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:35.907311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:35.907311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:35.907311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:35.907311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:35.907311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:35.907311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:35.907311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:35.907311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:35.907311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:35.907311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:35.907311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:35.907311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:35.907311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:35.908172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:35.908172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:35.908172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:35.908172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:35.908172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:35.908172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:35.908172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:35.908172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:35.908172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:35.908172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:35.908172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:35.908172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:35.908172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:35.908172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:35.908172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:35.908172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:35.909017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:35.909017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:35.909017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:35.909017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:35.909017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:35.909017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:35.909017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:35.909017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:35.909017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:35.909017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:35.909017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:35.909017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:35.912812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 200/399] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:35.918709 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f17dee-19e6-4298-a3a5-f46b99c4e35c None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:35.979448 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a322a43-d00f-460e-bb37-bc2538da18cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:35.980656 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a322a43-d00f-460e-bb37-bc2538da18cd None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:35.981145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a322a43-d00f-460e-bb37-bc2538da18cd None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:35.981636 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a322a43-d00f-460e-bb37-bc2538da18cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.005551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a322a43-d00f-460e-bb37-bc2538da18cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:36.014219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-9a322a43-d00f-460e-bb37-bc2538da18cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:36.014219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:36.014219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:36.014219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:36.014219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:36.014219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:36.014219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:36.014219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:36.014219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:36.014219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:36.014219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:36.014219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:36.014219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:36.014219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:36.014219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:36.014219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:36.015088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:36.015088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:36.015088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:36.015088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:36.015088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:36.015088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:36.015088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:36.015088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:36.015088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:36.015088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:36.015088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:36.015088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:36.015088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:36.015088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:36.015088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:36.015088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:36.015986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:36.015986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:36.015986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:36.015986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:36.015986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:36.015986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:36.015986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:36.015986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:36.015986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:36.015986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:36.015986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:36.015986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:36.030462 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 200/400] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:36.032099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a322a43-d00f-460e-bb37-bc2538da18cd None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.088614 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f17dee-19e6-4298-a3a5-f46b99c4e35c None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.130340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71997501-ed67-4f63-89ec-77299e17b3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.131419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71997501-ed67-4f63-89ec-77299e17b3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.131908 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71997501-ed67-4f63-89ec-77299e17b3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.132431 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71997501-ed67-4f63-89ec-77299e17b3c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.133419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71997501-ed67-4f63-89ec-77299e17b3c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:36.133931 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71997501-ed67-4f63-89ec-77299e17b3c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:36.135477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71997501-ed67-4f63-89ec-77299e17b3c4 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:36.145020 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 201/401] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:36.150848 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71997501-ed67-4f63-89ec-77299e17b3c4 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.160983 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:36.223270 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a322a43-d00f-460e-bb37-bc2538da18cd None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.260833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71997501-ed67-4f63-89ec-77299e17b3c4 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.286558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3dd5b-7c38-4173-bfd7-cb233360f858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.287415 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3dd5b-7c38-4173-bfd7-cb233360f858 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.287648 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3dd5b-7c38-4173-bfd7-cb233360f858 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.287921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3dd5b-7c38-4173-bfd7-cb233360f858 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.288757 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3dd5b-7c38-4173-bfd7-cb233360f858 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:36.289003 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3dd5b-7c38-4173-bfd7-cb233360f858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:36.290520 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3dd5b-7c38-4173-bfd7-cb233360f858 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:36.303701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fc69c-ed7a-4ca5-b4e8-0b47fe48b98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.305015 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fc69c-ed7a-4ca5-b4e8-0b47fe48b98d None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.306569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fc69c-ed7a-4ca5-b4e8-0b47fe48b98d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.307068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fc69c-ed7a-4ca5-b4e8-0b47fe48b98d None admin] PATH_INFO: `/v3/groups/0719110570b54fdf93607daf15940577` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.310257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 201/402] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:36.315612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fc69c-ed7a-4ca5-b4e8-0b47fe48b98d None admin] RBAC: Authorizing `identity:get_group(group_id=0719110570b54fdf93607daf15940577)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:36.317349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fc69c-ed7a-4ca5-b4e8-0b47fe48b98d None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:36.320190 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 202/403] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:36 2022] GET /identity/v3/groups/0719110570b54fdf93607daf15940577 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:36.321775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3dd5b-7c38-4173-bfd7-cb233360f858 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.325525 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fc69c-ed7a-4ca5-b4e8-0b47fe48b98d None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.327750 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:36.476755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:36.524517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3dd5b-7c38-4173-bfd7-cb233360f858 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.539006 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fc69c-ed7a-4ca5-b4e8-0b47fe48b98d None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.575233 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54be9c-23f0-4b6c-be9f-cddf5bd5dcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.576104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54be9c-23f0-4b6c-be9f-cddf5bd5dcea None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.576412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54be9c-23f0-4b6c-be9f-cddf5bd5dcea None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.576692 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54be9c-23f0-4b6c-be9f-cddf5bd5dcea None admin] PATH_INFO: `/v3/groups/0719110570b54fdf93607daf15940577` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.577906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54be9c-23f0-4b6c-be9f-cddf5bd5dcea None admin] RBAC: Authorizing `identity:get_group(group_id=0719110570b54fdf93607daf15940577)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:36.579365 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54be9c-23f0-4b6c-be9f-cddf5bd5dcea None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:36.581313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 202/404] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:36 2022] GET /identity/v3/groups/0719110570b54fdf93607daf15940577 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:36.582096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357cf06-2ecd-46c2-8710-cda7e6cab0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.582478 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b910446-f1cb-4c71-b696-b65bce9c662e None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.582767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b910446-f1cb-4c71-b696-b65bce9c662e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.583048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b910446-f1cb-4c71-b696-b65bce9c662e None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.583701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357cf06-2ecd-46c2-8710-cda7e6cab0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.584732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357cf06-2ecd-46c2-8710-cda7e6cab0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.584834 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 203/405] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:36.585229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357cf06-2ecd-46c2-8710-cda7e6cab0c0 None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.586575 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a357cf06-2ecd-46c2-8710-cda7e6cab0c0 None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:36.586996 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b910446-f1cb-4c71-b696-b65bce9c662e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.592238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a357cf06-2ecd-46c2-8710-cda7e6cab0c0 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:36.594645 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 203/406] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:36 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:36.596073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5202316-1f90-45a5-91d3-c64bd39883fd None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.596563 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5202316-1f90-45a5-91d3-c64bd39883fd None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.597040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5202316-1f90-45a5-91d3-c64bd39883fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.615023 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-e5202316-1f90-45a5-91d3-c64bd39883fd None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:36.617097 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e5202316-1f90-45a5-91d3-c64bd39883fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:36.682758 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:36.703893 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 204/407] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:36.704874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5202316-1f90-45a5-91d3-c64bd39883fd None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.722242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b910446-f1cb-4c71-b696-b65bce9c662e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.768580 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac2550-10bf-42c1-92d5-1caa5f02d344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.769976 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac2550-10bf-42c1-92d5-1caa5f02d344 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.770443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac2550-10bf-42c1-92d5-1caa5f02d344 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.770899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac2550-10bf-42c1-92d5-1caa5f02d344 None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.772426 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac2550-10bf-42c1-92d5-1caa5f02d344 None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:36.774231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac2550-10bf-42c1-92d5-1caa5f02d344 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:36.781494 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 204/408] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:36 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:36.781494 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac2550-10bf-42c1-92d5-1caa5f02d344 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.793179 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:36.828075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5202316-1f90-45a5-91d3-c64bd39883fd None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.879893 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5e060-2346-464f-bbc9-8d0ff37d00f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.880831 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5e060-2346-464f-bbc9-8d0ff37d00f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.882206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5e060-2346-464f-bbc9-8d0ff37d00f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.882493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5e060-2346-464f-bbc9-8d0ff37d00f7 None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e/groups/0719110570b54fdf93607daf15940577/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.885377 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5e060-2346-464f-bbc9-8d0ff37d00f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=359a065a7d2c49fea870a82587a5d51e, group_id=0719110570b54fdf93607daf15940577, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:36.887061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5e060-2346-464f-bbc9-8d0ff37d00f7 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:36.911954 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 205/409] 10.222.0.43 () {58 vars in 1442 bytes} [Wed Jul 20 19:10:36 2022] PUT /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e/groups/0719110570b54fdf93607daf15940577/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:36.911954 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5e060-2346-464f-bbc9-8d0ff37d00f7 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.916783 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac2550-10bf-42c1-92d5-1caa5f02d344 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.000774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee909a9-e84c-40db-b58f-98e8bb6e4fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.001971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee909a9-e84c-40db-b58f-98e8bb6e4fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.002461 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee909a9-e84c-40db-b58f-98e8bb6e4fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.002957 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee909a9-e84c-40db-b58f-98e8bb6e4fe7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.025773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:37.029648 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5e060-2346-464f-bbc9-8d0ff37d00f7 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.033235 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee909a9-e84c-40db-b58f-98e8bb6e4fe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:37.052091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-6ee909a9-e84c-40db-b58f-98e8bb6e4fe7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:37.052091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:37.052091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:37.052091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:37.052091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:37.052091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:37.052091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:37.052091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:37.052091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:37.052091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:37.052091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:37.052091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:37.052091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:37.052091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:37.052091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:37.052091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:37.053748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:37.053748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:37.053748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:37.053748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:37.053748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:37.053748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:37.053748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:37.053748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:37.053748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:37.053748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:37.053748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:37.053748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:37.053748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:37.053748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:37.053748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:37.053748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:37.054833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:37.054833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:37.054833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:37.054833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:37.054833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:37.054833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:37.054833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:37.054833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:37.054833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:37.054833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:37.054833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:37.054833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:37.054833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 205/410] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:37.060776 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee909a9-e84c-40db-b58f-98e8bb6e4fe7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:37.074542 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f11362-151d-4120-8960-40df2034f85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.076791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f11362-151d-4120-8960-40df2034f85b None admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.077293 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f11362-151d-4120-8960-40df2034f85b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.077753 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f11362-151d-4120-8960-40df2034f85b None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e/groups/0719110570b54fdf93607daf15940577/roles/abe200f23fc84ba397dd8c67874f0f2a` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.080412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f11362-151d-4120-8960-40df2034f85b None admin] RBAC: Authorizing `identity:create_grant(project_id=359a065a7d2c49fea870a82587a5d51e, group_id=0719110570b54fdf93607daf15940577, role_id=abe200f23fc84ba397dd8c67874f0f2a)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:37.082802 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f11362-151d-4120-8960-40df2034f85b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:37.088769 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:37.111684 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 206/411] 10.222.0.43 () {58 vars in 1442 bytes} [Wed Jul 20 19:10:36 2022] PUT /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e/groups/0719110570b54fdf93607daf15940577/roles/abe200f23fc84ba397dd8c67874f0f2a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:37.249417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee909a9-e84c-40db-b58f-98e8bb6e4fe7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.287447 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65b1b9-51aa-4622-8a23-f3d79fe51e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.288046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65b1b9-51aa-4622-8a23-f3d79fe51e30 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.288396 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65b1b9-51aa-4622-8a23-f3d79fe51e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.288638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65b1b9-51aa-4622-8a23-f3d79fe51e30 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.289404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65b1b9-51aa-4622-8a23-f3d79fe51e30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:37.289708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65b1b9-51aa-4622-8a23-f3d79fe51e30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:37.291085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65b1b9-51aa-4622-8a23-f3d79fe51e30 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:37.301005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 206/412] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:37.306763 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f11362-151d-4120-8960-40df2034f85b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:37.315475 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7298ff-06ff-44d7-84ca-0e29902b3a6d None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.315782 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7298ff-06ff-44d7-84ca-0e29902b3a6d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.316049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7298ff-06ff-44d7-84ca-0e29902b3a6d None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.317465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 207/413] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:37.322771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5a138-0bb3-4954-90b5-10835de8aae1 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.323060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5a138-0bb3-4954-90b5-10835de8aae1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.323338 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5a138-0bb3-4954-90b5-10835de8aae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.345011 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c2c5a138-0bb3-4954-90b5-10835de8aae1 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:37.346367 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c2c5a138-0bb3-4954-90b5-10835de8aae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:37.436553 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:37.455255 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 208/414] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:37.473363 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe19a4-2e41-4f8b-934e-6a6193ec1015 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.473746 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe19a4-2e41-4f8b-934e-6a6193ec1015 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.474101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe19a4-2e41-4f8b-934e-6a6193ec1015 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.476418 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 209/415] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:37.482255 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd202331-2bf6-4a57-bfde-10b27439c348 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.483118 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd202331-2bf6-4a57-bfde-10b27439c348 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.483423 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd202331-2bf6-4a57-bfde-10b27439c348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.506063 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-cd202331-2bf6-4a57-bfde-10b27439c348 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:37.508076 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cd202331-2bf6-4a57-bfde-10b27439c348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:37.522030 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f11362-151d-4120-8960-40df2034f85b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.558420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6e1c8-6a42-47fe-b621-0d636302d741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.559541 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6e1c8-6a42-47fe-b621-0d636302d741 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.559853 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6e1c8-6a42-47fe-b621-0d636302d741 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.560122 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6e1c8-6a42-47fe-b621-0d636302d741 None admin] PATH_INFO: `/v3/groups/3bde401d70ac4c90a2c020d6c7c337a9` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.561388 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6e1c8-6a42-47fe-b621-0d636302d741 None admin] RBAC: Authorizing `identity:get_group(group_id=3bde401d70ac4c90a2c020d6c7c337a9)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:37.562830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6e1c8-6a42-47fe-b621-0d636302d741 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:37.564663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 207/416] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:37 2022] GET /identity/v3/groups/3bde401d70ac4c90a2c020d6c7c337a9 => generated 229 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:37.565429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6e1c8-6a42-47fe-b621-0d636302d741 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:37.590203 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 210/417] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:37.591090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd202331-2bf6-4a57-bfde-10b27439c348 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:37.650727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:37.711183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6e1c8-6a42-47fe-b621-0d636302d741 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.719071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd202331-2bf6-4a57-bfde-10b27439c348 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.743458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:37.758807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50748eb-4867-44b6-b456-2440dbe8c2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.759069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acff79a-ef09-4422-a827-46c2cc4d18f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.759914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50748eb-4867-44b6-b456-2440dbe8c2fe None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.760212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acff79a-ef09-4422-a827-46c2cc4d18f3 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.760390 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50748eb-4867-44b6-b456-2440dbe8c2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.760764 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50748eb-4867-44b6-b456-2440dbe8c2fe None admin] PATH_INFO: `/v3/projects/39f878600fda47389da52621d6e0947c` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.760942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acff79a-ef09-4422-a827-46c2cc4d18f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.761373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acff79a-ef09-4422-a827-46c2cc4d18f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.762226 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50748eb-4867-44b6-b456-2440dbe8c2fe None admin] RBAC: Authorizing `identity:get_project(project_id=39f878600fda47389da52621d6e0947c)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:37.764033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50748eb-4867-44b6-b456-2440dbe8c2fe None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:37.766598 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 211/418] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:37 2022] GET /identity/v3/projects/39f878600fda47389da52621d6e0947c => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:37.767778 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50748eb-4867-44b6-b456-2440dbe8c2fe None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:37.773949 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acff79a-ef09-4422-a827-46c2cc4d18f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:37.782088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-7acff79a-ef09-4422-a827-46c2cc4d18f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:37.782088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:37.782088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:37.782088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:37.782088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:37.782088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:37.782088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:37.782088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:37.782088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:37.782088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:37.782088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:37.782088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:37.782088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:37.782088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:37.782088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:37.782088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:37.782972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:37.782972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:37.782972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:37.782972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:37.782972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:37.782972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:37.782972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:37.782972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:37.782972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:37.782972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:37.782972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:37.782972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:37.782972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:37.782972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:37.782972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:37.782972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:37.783875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:37.783875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:37.783875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:37.783875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:37.783875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:37.783875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:37.783875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:37.783875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:37.783875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:37.783875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:37.783875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:37.783875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:37.786646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 208/419] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:37.787757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acff79a-ef09-4422-a827-46c2cc4d18f3 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:37.899456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acff79a-ef09-4422-a827-46c2cc4d18f3 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.900224 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50748eb-4867-44b6-b456-2440dbe8c2fe None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.921841 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:37.938261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c5a42-edd4-450a-af7f-f120940cc78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.939488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c5a42-edd4-450a-af7f-f120940cc78a None admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.939976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c5a42-edd4-450a-af7f-f120940cc78a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.940476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c5a42-edd4-450a-af7f-f120940cc78a None admin] PATH_INFO: `/v3/projects/39f878600fda47389da52621d6e0947c/groups/3bde401d70ac4c90a2c020d6c7c337a9/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.943070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c5a42-edd4-450a-af7f-f120940cc78a None admin] RBAC: Authorizing `identity:create_grant(project_id=39f878600fda47389da52621d6e0947c, group_id=3bde401d70ac4c90a2c020d6c7c337a9, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:37.945187 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c5a42-edd4-450a-af7f-f120940cc78a None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:37.945630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3a9fd-55cd-41bf-b2bf-91209770e5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.946447 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3a9fd-55cd-41bf-b2bf-91209770e5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.946732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3a9fd-55cd-41bf-b2bf-91209770e5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.947011 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3a9fd-55cd-41bf-b2bf-91209770e5a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.961015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3a9fd-55cd-41bf-b2bf-91209770e5a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:37.963023 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 209/420] 10.222.0.43 () {58 vars in 1442 bytes} [Wed Jul 20 19:10:37 2022] PUT /identity/v3/projects/39f878600fda47389da52621d6e0947c/groups/3bde401d70ac4c90a2c020d6c7c337a9/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:37.964165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c5a42-edd4-450a-af7f-f120940cc78a None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:37.969790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-9fc3a9fd-55cd-41bf-b2bf-91209770e5a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:37.969790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:37.969790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:37.969790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:37.969790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:37.969790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:37.969790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:37.969790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:37.969790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:37.969790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:37.969790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:37.969790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:37.969790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:37.969790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:37.969790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:37.969790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:37.970770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:37.970770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:37.970770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:37.970770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:37.970770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:37.970770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:37.970770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:37.970770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:37.970770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:37.970770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:37.970770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:37.970770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:37.970770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:37.970770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:37.970770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:37.970770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:37.972062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:37.972062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:37.972062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:37.972062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:37.972062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:37.972062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:37.972062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:37.972062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:37.972062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:37.972062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:37.972062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:37.972062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:37.976205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 212/421] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:37.977225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3a9fd-55cd-41bf-b2bf-91209770e5a3 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:38.032686 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:38.090595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c5a42-edd4-450a-af7f-f120940cc78a None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:38.090905 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3a9fd-55cd-41bf-b2bf-91209770e5a3 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:38.129359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783f9c4-52a7-4b4c-b026-9bfd680163e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:38.130296 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783f9c4-52a7-4b4c-b026-9bfd680163e3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:38.130879 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783f9c4-52a7-4b4c-b026-9bfd680163e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:38.131340 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783f9c4-52a7-4b4c-b026-9bfd680163e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:38.132379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783f9c4-52a7-4b4c-b026-9bfd680163e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:38.132838 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783f9c4-52a7-4b4c-b026-9bfd680163e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:38.134373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783f9c4-52a7-4b4c-b026-9bfd680163e3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:38.143905 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 213/422] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:38.149964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783f9c4-52a7-4b4c-b026-9bfd680163e3 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:38.177723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8b4ac-65c0-4589-8ab7-db5ff0f95b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:38.178430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8b4ac-65c0-4589-8ab7-db5ff0f95b32 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:38.178719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8b4ac-65c0-4589-8ab7-db5ff0f95b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:38.178996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8b4ac-65c0-4589-8ab7-db5ff0f95b32 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:38.179756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8b4ac-65c0-4589-8ab7-db5ff0f95b32 None admin] RBAC: Adding query filter params (name=member) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:38.180062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8b4ac-65c0-4589-8ab7-db5ff0f95b32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:38.182018 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8b4ac-65c0-4589-8ab7-db5ff0f95b32 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:38.191316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 210/423] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:38.197260 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8b4ac-65c0-4589-8ab7-db5ff0f95b32 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:38.250592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:38.283365 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783f9c4-52a7-4b4c-b026-9bfd680163e3 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:38.297242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:38.316743 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8b4ac-65c0-4589-8ab7-db5ff0f95b32 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:38.320025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c705b-5ef2-41c2-b908-db151d5706c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:38.321069 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c705b-5ef2-41c2-b908-db151d5706c5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:38.321069 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c705b-5ef2-41c2-b908-db151d5706c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:38.321472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c705b-5ef2-41c2-b908-db151d5706c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:38.355922 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1336b-2235-4af1-957b-907ebb4bb3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:38.356657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1336b-2235-4af1-957b-907ebb4bb3ba None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:38.356951 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1336b-2235-4af1-957b-907ebb4bb3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:38.357234 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1336b-2235-4af1-957b-907ebb4bb3ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:38.431270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1336b-2235-4af1-957b-907ebb4bb3ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:38.432717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1336b-2235-4af1-957b-907ebb4bb3ba None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:38.434000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c705b-5ef2-41c2-b908-db151d5706c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:38.435745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c705b-5ef2-41c2-b908-db151d5706c5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:38.506600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:38.546370 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 211/424] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:38.553828 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1336b-2235-4af1-957b-907ebb4bb3ba None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:38.659138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1336b-2235-4af1-957b-907ebb4bb3ba None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:38.694875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53fe0c4-91d4-4cdc-85c0-fe21628541bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:38.695666 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fe0c4-91d4-4cdc-85c0-fe21628541bb None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:38.695956 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fe0c4-91d4-4cdc-85c0-fe21628541bb None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:38.696233 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fe0c4-91d4-4cdc-85c0-fe21628541bb None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:38.697586 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53fe0c4-91d4-4cdc-85c0-fe21628541bb None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:38.699034 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53fe0c4-91d4-4cdc-85c0-fe21628541bb None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:38.701132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 212/425] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:38 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:38.707036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53fe0c4-91d4-4cdc-85c0-fe21628541bb None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:38.724073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:38.728242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 214/426] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:10:38.736787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c705b-5ef2-41c2-b908-db151d5706c5 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:38.762669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:38.843191 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53fe0c4-91d4-4cdc-85c0-fe21628541bb None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:38.879021 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e16831-26a5-4a95-bcf9-ef71457181f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:38.879906 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e16831-26a5-4a95-bcf9-ef71457181f5 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:38.880190 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e16831-26a5-4a95-bcf9-ef71457181f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:38.880500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e16831-26a5-4a95-bcf9-ef71457181f5 None admin] PATH_INFO: `/v3/groups/0719110570b54fdf93607daf15940577` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:38.882244 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e16831-26a5-4a95-bcf9-ef71457181f5 None admin] RBAC: Authorizing `identity:get_group(group_id=0719110570b54fdf93607daf15940577)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:38.883690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e16831-26a5-4a95-bcf9-ef71457181f5 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:38.885666 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 213/427] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:38 2022] GET /identity/v3/groups/0719110570b54fdf93607daf15940577 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:38.891798 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e16831-26a5-4a95-bcf9-ef71457181f5 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:38.985305 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:39.011776 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e16831-26a5-4a95-bcf9-ef71457181f5 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:39.022483 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c705b-5ef2-41c2-b908-db151d5706c5 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:39.052330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9dff7-72a1-41cc-a53a-1a97d63a5ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:39.053431 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9dff7-72a1-41cc-a53a-1a97d63a5ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.053951 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9dff7-72a1-41cc-a53a-1a97d63a5ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.054454 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9dff7-72a1-41cc-a53a-1a97d63a5ed0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.055497 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9dff7-72a1-41cc-a53a-1a97d63a5ed0 None admin] RBAC: Adding query filter params (group.id=0719110570b54fdf93607daf15940577, role.id=18f0dc41b78c4b52b6912225928ea388, scope.project.id=711e8b07480d478a80cbefc6fe30467f) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:39.056018 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9dff7-72a1-41cc-a53a-1a97d63a5ed0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:39.058206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9dff7-72a1-41cc-a53a-1a97d63a5ed0 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:39.066050 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 214/428] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:38 2022] GET /identity/v3/role_assignments?group.id=0719110570b54fdf93607daf15940577&scope.project.id=711e8b07480d478a80cbefc6fe30467f&role.id=18f0dc41b78c4b52b6912225928ea388 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:39.092084 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e52bf-2cbb-4541-a182-1dc6f9d31ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:39.093386 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e52bf-2cbb-4541-a182-1dc6f9d31ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.093874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e52bf-2cbb-4541-a182-1dc6f9d31ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.094353 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e52bf-2cbb-4541-a182-1dc6f9d31ac7 None admin] PATH_INFO: `/v3/projects/711e8b07480d478a80cbefc6fe30467f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.101285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e52bf-2cbb-4541-a182-1dc6f9d31ac7 None admin] RBAC: Authorizing `identity:get_project(project_id=711e8b07480d478a80cbefc6fe30467f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:39.103161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e52bf-2cbb-4541-a182-1dc6f9d31ac7 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:39.105506 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 215/429] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:38 2022] GET /identity/v3/projects/711e8b07480d478a80cbefc6fe30467f => generated 298 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:39.111372 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9dff7-72a1-41cc-a53a-1a97d63a5ed0 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:39.236719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9dff7-72a1-41cc-a53a-1a97d63a5ed0 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:39.276841 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114d79c-3556-4079-9767-10bd2ff30dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:39.277717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114d79c-3556-4079-9767-10bd2ff30dec None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.278000 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114d79c-3556-4079-9767-10bd2ff30dec None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.278274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114d79c-3556-4079-9767-10bd2ff30dec None admin] PATH_INFO: `/v3/groups/0719110570b54fdf93607daf15940577` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.279468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114d79c-3556-4079-9767-10bd2ff30dec None admin] RBAC: Authorizing `identity:get_group(group_id=0719110570b54fdf93607daf15940577)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:39.280998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114d79c-3556-4079-9767-10bd2ff30dec None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:39.282951 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 215/430] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:39 2022] GET /identity/v3/groups/0719110570b54fdf93607daf15940577 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:39.288461 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e52bf-2cbb-4541-a182-1dc6f9d31ac7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:39.397225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:39.411516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e52bf-2cbb-4541-a182-1dc6f9d31ac7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:39.452993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1f45f-abba-4729-bb7d-ac49d8e39fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:39.453669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1f45f-abba-4729-bb7d-ac49d8e39fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.453994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1f45f-abba-4729-bb7d-ac49d8e39fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.454279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1f45f-abba-4729-bb7d-ac49d8e39fc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.455138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1f45f-abba-4729-bb7d-ac49d8e39fc7 None admin] RBAC: Adding query filter params (group.id=0719110570b54fdf93607daf15940577, role.id=abe200f23fc84ba397dd8c67874f0f2a, scope.project.id=711e8b07480d478a80cbefc6fe30467f) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:39.455454 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1f45f-abba-4729-bb7d-ac49d8e39fc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:39.456993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1f45f-abba-4729-bb7d-ac49d8e39fc7 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:39.468611 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 216/431] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:39 2022] GET /identity/v3/role_assignments?group.id=0719110570b54fdf93607daf15940577&scope.project.id=711e8b07480d478a80cbefc6fe30467f&role.id=abe200f23fc84ba397dd8c67874f0f2a => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:39.536458 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f389af-a0d7-49a0-ab75-dc629e574312 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.536757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f389af-a0d7-49a0-ab75-dc629e574312 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.537002 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f389af-a0d7-49a0-ab75-dc629e574312 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.538862 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 216/432] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:39.544280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22955cb4-1e68-4bc8-9a5b-c32811d27c32 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.544595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22955cb4-1e68-4bc8-9a5b-c32811d27c32 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.544876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22955cb4-1e68-4bc8-9a5b-c32811d27c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.554057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731846d0-0eb9-4148-bcd0-bda3f6546c3c None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.554402 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731846d0-0eb9-4148-bcd0-bda3f6546c3c None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.554623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731846d0-0eb9-4148-bcd0-bda3f6546c3c None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.556323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 217/433] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:39.562522 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbb3e1-a5d5-40a4-a666-649ef7b29b5c None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.562816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbb3e1-a5d5-40a4-a666-649ef7b29b5c None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.563073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbb3e1-a5d5-40a4-a666-649ef7b29b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.566780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-22955cb4-1e68-4bc8-9a5b-c32811d27c32 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:39.568146 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-22955cb4-1e68-4bc8-9a5b-c32811d27c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:39.589572 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-3cfbb3e1-a5d5-40a4-a666-649ef7b29b5c None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:39.590791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-3cfbb3e1-a5d5-40a4-a666-649ef7b29b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:39.599748 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:39.659205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 217/434] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:39.674527 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 218/435] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:39.685528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22955cb4-1e68-4bc8-9a5b-c32811d27c32 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:39.700325 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbb3e1-a5d5-40a4-a666-649ef7b29b5c None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:39.780152 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:39.813881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22955cb4-1e68-4bc8-9a5b-c32811d27c32 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:39.830141 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbb3e1-a5d5-40a4-a666-649ef7b29b5c None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:39.865768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3abb0-fca4-47c1-a77b-7c4f73888674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:39.866582 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3abb0-fca4-47c1-a77b-7c4f73888674 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.866867 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3abb0-fca4-47c1-a77b-7c4f73888674 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.867139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3abb0-fca4-47c1-a77b-7c4f73888674 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.874216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c9b90-a9bd-4624-b62d-74703cf4307f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:39.875238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c9b90-a9bd-4624-b62d-74703cf4307f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.875665 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c9b90-a9bd-4624-b62d-74703cf4307f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.876109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c9b90-a9bd-4624-b62d-74703cf4307f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.883853 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3abb0-fca4-47c1-a77b-7c4f73888674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:39.886378 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:39.890521 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c9b90-a9bd-4624-b62d-74703cf4307f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:39.893947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-5fd3abb0-fca4-47c1-a77b-7c4f73888674 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:39.893947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:39.893947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:39.893947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:39.893947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:39.893947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:39.893947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:39.893947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:39.893947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:39.893947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:39.893947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:39.893947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:39.893947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:39.893947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:39.893947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:39.893947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:39.894827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:39.894827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:39.894827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:39.894827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:39.894827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:39.894827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:39.894827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:39.894827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:39.894827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:39.894827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:39.894827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:39.894827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:39.894827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:39.894827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:39.894827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:39.894827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:39.895595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:39.895595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:39.895595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:39.895595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:39.895595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:39.895595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:39.895595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:39.895595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:39.895595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:39.895595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:39.895595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:39.895595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:39.895595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 218/436] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:39.900823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3abb0-fca4-47c1-a77b-7c4f73888674 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:39.902935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-791c9b90-a9bd-4624-b62d-74703cf4307f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:39.902935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:39.902935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:39.902935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:39.902935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:39.902935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:39.902935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:39.902935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:39.902935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:39.902935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:39.902935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:39.902935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:39.902935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:39.902935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:39.902935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:39.902935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:39.903810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:39.903810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:39.903810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:39.903810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:39.903810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:39.903810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:39.903810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:39.903810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:39.903810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:39.903810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:39.903810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:39.903810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:39.903810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:39.903810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:39.903810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:39.903810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:39.904647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:39.904647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:39.904647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:39.904647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:39.904647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:39.904647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:39.904647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:39.904647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:39.904647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:39.904647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:39.904647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:39.904647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:39.905898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 219/437] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:39.911660 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c9b90-a9bd-4624-b62d-74703cf4307f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.042328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c9b90-a9bd-4624-b62d-74703cf4307f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:40.045210 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3abb0-fca4-47c1-a77b-7c4f73888674 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:40.082104 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428cfab-04a9-4920-a072-d0522661bb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:40.082696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428cfab-04a9-4920-a072-d0522661bb36 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.082979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428cfab-04a9-4920-a072-d0522661bb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.083252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428cfab-04a9-4920-a072-d0522661bb36 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.084609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428cfab-04a9-4920-a072-d0522661bb36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:40.084916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428cfab-04a9-4920-a072-d0522661bb36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:40.086264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428cfab-04a9-4920-a072-d0522661bb36 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:40.088003 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65b0fa-b9a7-48bf-b357-a2401989a843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:40.089249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65b0fa-b9a7-48bf-b357-a2401989a843 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.089762 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65b0fa-b9a7-48bf-b357-a2401989a843 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.090231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65b0fa-b9a7-48bf-b357-a2401989a843 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.091617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a65b0fa-b9a7-48bf-b357-a2401989a843 None admin] RBAC: Adding query filter params (name=member) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:40.092145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a65b0fa-b9a7-48bf-b357-a2401989a843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:40.093732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a65b0fa-b9a7-48bf-b357-a2401989a843 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:40.094428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 220/438] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:40.095987 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:40.100576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428cfab-04a9-4920-a072-d0522661bb36 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.105985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 219/439] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:40.111202 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65b0fa-b9a7-48bf-b357-a2401989a843 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.170363 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:40.214718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428cfab-04a9-4920-a072-d0522661bb36 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:40.224331 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65b0fa-b9a7-48bf-b357-a2401989a843 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:40.249306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae37aa4-6d21-4786-ae8a-695267f369c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:40.249958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae37aa4-6d21-4786-ae8a-695267f369c3 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.250241 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae37aa4-6d21-4786-ae8a-695267f369c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.250522 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae37aa4-6d21-4786-ae8a-695267f369c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.261941 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8114546-4606-4be5-8928-cc06cdb1faf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:40.262593 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8114546-4606-4be5-8928-cc06cdb1faf5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.263201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8114546-4606-4be5-8928-cc06cdb1faf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.263484 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8114546-4606-4be5-8928-cc06cdb1faf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.322873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae37aa4-6d21-4786-ae8a-695267f369c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:40.324596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae37aa4-6d21-4786-ae8a-695267f369c3 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:40.330201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:40.339400 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8114546-4606-4be5-8928-cc06cdb1faf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:40.340827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8114546-4606-4be5-8928-cc06cdb1faf5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:40.396847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:40.439402 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 221/440] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:10:40.440818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56382c22-954a-47c7-be12-b7526e50fb91 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.441116 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56382c22-954a-47c7-be12-b7526e50fb91 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.441397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56382c22-954a-47c7-be12-b7526e50fb91 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.442820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 222/441] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:40.445049 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56382c22-954a-47c7-be12-b7526e50fb91 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.460083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 220/442] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:10:40.461416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7af829-7512-4702-a256-b7f2242b90fd None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.461711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7af829-7512-4702-a256-b7f2242b90fd None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.461989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7af829-7512-4702-a256-b7f2242b90fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.478019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0d7af829-7512-4702-a256-b7f2242b90fd None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:40.479206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0d7af829-7512-4702-a256-b7f2242b90fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:40.543970 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56382c22-954a-47c7-be12-b7526e50fb91 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:40.544311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 221/443] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:40.545330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7af829-7512-4702-a256-b7f2242b90fd None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.574473 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:40.586755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54474cd2-c575-42b2-a710-f4675ac1bd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:40.587620 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54474cd2-c575-42b2-a710-f4675ac1bd96 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.587906 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54474cd2-c575-42b2-a710-f4675ac1bd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.588189 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54474cd2-c575-42b2-a710-f4675ac1bd96 None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.589457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54474cd2-c575-42b2-a710-f4675ac1bd96 None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:40.590826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54474cd2-c575-42b2-a710-f4675ac1bd96 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:40.592902 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 223/444] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:40 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:40.593389 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54474cd2-c575-42b2-a710-f4675ac1bd96 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.622710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:40.650363 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7af829-7512-4702-a256-b7f2242b90fd None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:40.685922 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba504943-6e75-4d7c-9ec2-194952542a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:40.686767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba504943-6e75-4d7c-9ec2-194952542a74 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.687046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba504943-6e75-4d7c-9ec2-194952542a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.687317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba504943-6e75-4d7c-9ec2-194952542a74 None admin] PATH_INFO: `/v3/projects/359a065a7d2c49fea870a82587a5d51e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.688598 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba504943-6e75-4d7c-9ec2-194952542a74 None admin] RBAC: Authorizing `identity:get_project(project_id=359a065a7d2c49fea870a82587a5d51e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:40.689936 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba504943-6e75-4d7c-9ec2-194952542a74 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:40.691605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 222/445] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:40 2022] GET /identity/v3/projects/359a065a7d2c49fea870a82587a5d51e => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:40.692393 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba504943-6e75-4d7c-9ec2-194952542a74 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.692788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54474cd2-c575-42b2-a710-f4675ac1bd96 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:40.731982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ec9d5-7916-45e6-be68-aa27019915e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:40.732818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ec9d5-7916-45e6-be68-aa27019915e1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.733103 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ec9d5-7916-45e6-be68-aa27019915e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.733373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ec9d5-7916-45e6-be68-aa27019915e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.743218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ec9d5-7916-45e6-be68-aa27019915e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:40.751001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-926ec9d5-7916-45e6-be68-aa27019915e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:40.751001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:40.751001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:40.751001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:40.751001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:40.751001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:40.751001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:40.751001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:40.751001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:40.751001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:40.751001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:40.751001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:40.751001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:40.751001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:40.751001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:40.751001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:40.751863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:40.751863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:40.751863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:40.751863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:40.751863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:40.751863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:40.751863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:40.751863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:40.751863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:40.751863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:40.751863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:40.751863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:40.751863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:40.751863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:40.751863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:40.751863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:40.752842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:40.752842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:40.752842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:40.752842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:40.752842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:40.752842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:40.752842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:40.752842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:40.752842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:40.752842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:40.752842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:40.752842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:40.752842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 224/446] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:40.753716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ec9d5-7916-45e6-be68-aa27019915e1 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.790239 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:40.797465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba504943-6e75-4d7c-9ec2-194952542a74 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:40.833165 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a392106-460c-456c-9923-5452a4964bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:40.834495 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a392106-460c-456c-9923-5452a4964bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.834773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a392106-460c-456c-9923-5452a4964bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.835041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a392106-460c-456c-9923-5452a4964bc6 None admin] PATH_INFO: `/v3/groups/0719110570b54fdf93607daf15940577` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.836207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a392106-460c-456c-9923-5452a4964bc6 None admin] RBAC: Authorizing `identity:get_group(group_id=0719110570b54fdf93607daf15940577)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:40.837685 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a392106-460c-456c-9923-5452a4964bc6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:40.839290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 223/447] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:40 2022] GET /identity/v3/groups/0719110570b54fdf93607daf15940577 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:40.840102 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a392106-460c-456c-9923-5452a4964bc6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.842938 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:40.853482 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ec9d5-7916-45e6-be68-aa27019915e1 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:40.886887 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438a3cf-a9ed-4605-9810-7b2f97bd69f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:40.887741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438a3cf-a9ed-4605-9810-7b2f97bd69f1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.888030 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438a3cf-a9ed-4605-9810-7b2f97bd69f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.888332 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438a3cf-a9ed-4605-9810-7b2f97bd69f1 None admin] PATH_INFO: `/v3/groups/0719110570b54fdf93607daf15940577` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.889564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c438a3cf-a9ed-4605-9810-7b2f97bd69f1 None admin] RBAC: Authorizing `identity:get_group(group_id=0719110570b54fdf93607daf15940577)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:40.891002 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c438a3cf-a9ed-4605-9810-7b2f97bd69f1 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:40.892655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 225/448] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:40 2022] GET /identity/v3/groups/0719110570b54fdf93607daf15940577 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:40.893420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438a3cf-a9ed-4605-9810-7b2f97bd69f1 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.941809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a392106-460c-456c-9923-5452a4964bc6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:40.976537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa7b22-9580-4055-880a-1aa47041f021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:40.977201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa7b22-9580-4055-880a-1aa47041f021 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.977484 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa7b22-9580-4055-880a-1aa47041f021 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.977758 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa7b22-9580-4055-880a-1aa47041f021 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.978505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa7b22-9580-4055-880a-1aa47041f021 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:40.978811 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa7b22-9580-4055-880a-1aa47041f021 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:40.980191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa7b22-9580-4055-880a-1aa47041f021 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:40.988173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 224/449] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:40.988972 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa7b22-9580-4055-880a-1aa47041f021 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.992904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438a3cf-a9ed-4605-9810-7b2f97bd69f1 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:41.014937 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:41.029094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c78f75-6ed3-4d39-89c1-bc1aaee4a56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:41.029754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c78f75-6ed3-4d39-89c1-bc1aaee4a56f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:41.030262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c78f75-6ed3-4d39-89c1-bc1aaee4a56f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:41.030390 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c78f75-6ed3-4d39-89c1-bc1aaee4a56f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:41.031134 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c78f75-6ed3-4d39-89c1-bc1aaee4a56f None admin] RBAC: Adding query filter params (group.id=0719110570b54fdf93607daf15940577, role.id=abe200f23fc84ba397dd8c67874f0f2a, scope.project.id=359a065a7d2c49fea870a82587a5d51e) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:41.031456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c78f75-6ed3-4d39-89c1-bc1aaee4a56f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:41.032916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c78f75-6ed3-4d39-89c1-bc1aaee4a56f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:41.038709 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 226/450] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:40 2022] GET /identity/v3/role_assignments?group.id=0719110570b54fdf93607daf15940577&scope.project.id=359a065a7d2c49fea870a82587a5d51e&role.id=abe200f23fc84ba397dd8c67874f0f2a => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:41.039494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c78f75-6ed3-4d39-89c1-bc1aaee4a56f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:41.076187 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:41.103460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa7b22-9580-4055-880a-1aa47041f021 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:41.146843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db12b1-b0d4-4c58-96a4-dc43192b932a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:41.147511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db12b1-b0d4-4c58-96a4-dc43192b932a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:41.147797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db12b1-b0d4-4c58-96a4-dc43192b932a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:41.148074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db12b1-b0d4-4c58-96a4-dc43192b932a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:41.148991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db12b1-b0d4-4c58-96a4-dc43192b932a None admin] RBAC: Adding query filter params (group.id=0719110570b54fdf93607daf15940577, role.id=18f0dc41b78c4b52b6912225928ea388, scope.project.id=359a065a7d2c49fea870a82587a5d51e) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:41.149288 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db12b1-b0d4-4c58-96a4-dc43192b932a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:41.150807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db12b1-b0d4-4c58-96a4-dc43192b932a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:41.168336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 225/451] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:40 2022] GET /identity/v3/role_assignments?group.id=0719110570b54fdf93607daf15940577&scope.project.id=359a065a7d2c49fea870a82587a5d51e&role.id=18f0dc41b78c4b52b6912225928ea388 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:41.172500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c78f75-6ed3-4d39-89c1-bc1aaee4a56f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:41.219696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9b828-219e-4e08-9e4c-6495ef56cf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:41.220417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9b828-219e-4e08-9e4c-6495ef56cf10 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:41.220707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9b828-219e-4e08-9e4c-6495ef56cf10 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:41.220984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9b828-219e-4e08-9e4c-6495ef56cf10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:41.296428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9b828-219e-4e08-9e4c-6495ef56cf10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:41.297922 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9b828-219e-4e08-9e4c-6495ef56cf10 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:41.323132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:41.406871 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 227/452] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:41.412617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db12b1-b0d4-4c58-96a4-dc43192b932a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:41.502265 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:41.517855 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db12b1-b0d4-4c58-96a4-dc43192b932a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:41.554708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86375c-f50f-40ef-bf3d-2450f613b96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:41.555571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86375c-f50f-40ef-bf3d-2450f613b96f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:41.555853 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86375c-f50f-40ef-bf3d-2450f613b96f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:41.556129 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86375c-f50f-40ef-bf3d-2450f613b96f None admin] PATH_INFO: `/v3/projects/39f878600fda47389da52621d6e0947c` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:41.557355 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86375c-f50f-40ef-bf3d-2450f613b96f None admin] RBAC: Authorizing `identity:get_project(project_id=39f878600fda47389da52621d6e0947c)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:41.559047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86375c-f50f-40ef-bf3d-2450f613b96f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:41.560810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 226/453] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:41 2022] GET /identity/v3/projects/39f878600fda47389da52621d6e0947c => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:41.566730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9b828-219e-4e08-9e4c-6495ef56cf10 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:41.666660 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9b828-219e-4e08-9e4c-6495ef56cf10 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:41.689279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:41.701661 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e300d-79ec-4895-ac35-380cc99cd8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:41.702515 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e300d-79ec-4895-ac35-380cc99cd8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:41.702794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e300d-79ec-4895-ac35-380cc99cd8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:41.703063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e300d-79ec-4895-ac35-380cc99cd8e7 None admin] PATH_INFO: `/v3/groups/3bde401d70ac4c90a2c020d6c7c337a9` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:41.704211 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e300d-79ec-4895-ac35-380cc99cd8e7 None admin] RBAC: Authorizing `identity:get_group(group_id=3bde401d70ac4c90a2c020d6c7c337a9)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:41.705653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e300d-79ec-4895-ac35-380cc99cd8e7 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:41.707249 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 228/454] 10.222.0.43 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:41 2022] GET /identity/v3/groups/3bde401d70ac4c90a2c020d6c7c337a9 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:41.713129 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86375c-f50f-40ef-bf3d-2450f613b96f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:41.810054 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86375c-f50f-40ef-bf3d-2450f613b96f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:41.842044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54434aa-4fd2-4ba5-a973-5529779499c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:41.842678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54434aa-4fd2-4ba5-a973-5529779499c4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:41.842958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54434aa-4fd2-4ba5-a973-5529779499c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:41.843231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54434aa-4fd2-4ba5-a973-5529779499c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:41.844055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54434aa-4fd2-4ba5-a973-5529779499c4 None admin] RBAC: Adding query filter params (group.id=3bde401d70ac4c90a2c020d6c7c337a9, role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=39f878600fda47389da52621d6e0947c) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:41.844393 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54434aa-4fd2-4ba5-a973-5529779499c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:41.845807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54434aa-4fd2-4ba5-a973-5529779499c4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:41.851399 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 227/455] 10.222.0.43 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:41 2022] GET /identity/v3/role_assignments?group.id=3bde401d70ac4c90a2c020d6c7c337a9&scope.project.id=39f878600fda47389da52621d6e0947c&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:44.214438 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce29d49-a177-4d41-8964-71e0cf06dac4 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.215458 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce29d49-a177-4d41-8964-71e0cf06dac4 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.215962 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce29d49-a177-4d41-8964-71e0cf06dac4 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.217759 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 229/456] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:44.232802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2b466-6578-4633-94ab-b7ac59c721d2 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.233240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2b466-6578-4633-94ab-b7ac59c721d2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.233421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2b466-6578-4633-94ab-b7ac59c721d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.253273 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-bac2b466-6578-4633-94ab-b7ac59c721d2 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:44.254478 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bac2b466-6578-4633-94ab-b7ac59c721d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:44.273953 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:44.346977 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 228/457] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:44.379816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce29d49-a177-4d41-8964-71e0cf06dac4 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:44.460103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cc089-27f7-4a5e-b91d-9cd57e8632ed None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.460466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cc089-27f7-4a5e-b91d-9cd57e8632ed None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.460700 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cc089-27f7-4a5e-b91d-9cd57e8632ed None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.462452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 229/458] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:44.467706 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782596b4-c62f-49b7-bf5f-79b561f38e79 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.467940 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782596b4-c62f-49b7-bf5f-79b561f38e79 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.468222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782596b4-c62f-49b7-bf5f-79b561f38e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.493357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-782596b4-c62f-49b7-bf5f-79b561f38e79 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:44.494548 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-782596b4-c62f-49b7-bf5f-79b561f38e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:44.527173 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce29d49-a177-4d41-8964-71e0cf06dac4 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:44.575839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 230/459] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:44.586559 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c90791-0ee1-487b-b356-4acfc1403c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:44.587417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c90791-0ee1-487b-b356-4acfc1403c55 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.587699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c90791-0ee1-487b-b356-4acfc1403c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.587977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c90791-0ee1-487b-b356-4acfc1403c55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.601085 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c90791-0ee1-487b-b356-4acfc1403c55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:44.602543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c90791-0ee1-487b-b356-4acfc1403c55 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:44.618316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782596b4-c62f-49b7-bf5f-79b561f38e79 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:44.622875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-c1c90791-0ee1-487b-b356-4acfc1403c55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:44.622875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:44.622875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:44.622875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:44.622875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:44.622875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:44.622875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:44.622875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:44.622875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:44.622875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:44.622875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:44.622875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:44.622875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:44.622875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:44.622875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:44.623694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:44.624936 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:44.624936 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:44.624936 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:44.624936 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:44.624936 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 230/460] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:44.628721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c90791-0ee1-487b-b356-4acfc1403c55 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:44.674360 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:44.734846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c90791-0ee1-487b-b356-4acfc1403c55 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:44.772256 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03061c2a-b94a-4cf5-9785-2dffec2e9e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:44.772927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03061c2a-b94a-4cf5-9785-2dffec2e9e35 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.773205 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03061c2a-b94a-4cf5-9785-2dffec2e9e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.773476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03061c2a-b94a-4cf5-9785-2dffec2e9e35 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.774196 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03061c2a-b94a-4cf5-9785-2dffec2e9e35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:44.774505 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03061c2a-b94a-4cf5-9785-2dffec2e9e35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:44.775966 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03061c2a-b94a-4cf5-9785-2dffec2e9e35 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:44.788918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 231/461] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:44.794290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03061c2a-b94a-4cf5-9785-2dffec2e9e35 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:44.813282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:44.847515 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782596b4-c62f-49b7-bf5f-79b561f38e79 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:44.891931 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903fd5ae-ed8a-4360-a531-6170417654c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:44.893120 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fd5ae-ed8a-4360-a531-6170417654c6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.893605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fd5ae-ed8a-4360-a531-6170417654c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.894076 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fd5ae-ed8a-4360-a531-6170417654c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.901278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03061c2a-b94a-4cf5-9785-2dffec2e9e35 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:44.902579 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:44.905222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903fd5ae-ed8a-4360-a531-6170417654c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:44.907004 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903fd5ae-ed8a-4360-a531-6170417654c6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:44.926158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-903fd5ae-ed8a-4360-a531-6170417654c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:44.926158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:44.926158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:44.926158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:44.926158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:44.926158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:44.926158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:44.926158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:44.926158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:44.926158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:44.926158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:44.926158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:44.926158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:44.926158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:44.926158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:44.927045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:44.928007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:44.928007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:44.928007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:44.928007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:44.930302 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 231/462] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:44.931837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fce0ca-f65c-4ded-8b66-ec6eafcd87b6 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.932468 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fce0ca-f65c-4ded-8b66-ec6eafcd87b6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.932944 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fce0ca-f65c-4ded-8b66-ec6eafcd87b6 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.934972 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 232/463] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:44.936166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fce0ca-f65c-4ded-8b66-ec6eafcd87b6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:44.939136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab434d-834b-448a-8060-b971d6d91e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:44.940027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab434d-834b-448a-8060-b971d6d91e06 None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.940438 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab434d-834b-448a-8060-b971d6d91e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.941525 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab434d-834b-448a-8060-b971d6d91e06 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.941701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab434d-834b-448a-8060-b971d6d91e06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:44.943751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab434d-834b-448a-8060-b971d6d91e06 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:44.966681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 232/464] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:10:44 2022] POST /identity/v3/users => generated 246 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:10:44.974107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dce69-21f3-4bf6-a451-c4f99e333c78 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.975174 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dce69-21f3-4bf6-a451-c4f99e333c78 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.975551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dce69-21f3-4bf6-a451-c4f99e333c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.000580 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-d95dce69-21f3-4bf6-a451-c4f99e333c78 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:45.002094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d95dce69-21f3-4bf6-a451-c4f99e333c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:45.051122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fce0ca-f65c-4ded-8b66-ec6eafcd87b6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:45.086089 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 233/465] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:45.099017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:45.106399 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eef971-10d5-4c62-923f-5753d29f6c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:45.107329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eef971-10d5-4c62-923f-5753d29f6c33 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.107755 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eef971-10d5-4c62-923f-5753d29f6c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.108203 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eef971-10d5-4c62-923f-5753d29f6c33 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.109105 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eef971-10d5-4c62-923f-5753d29f6c33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:45.109570 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eef971-10d5-4c62-923f-5753d29f6c33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:45.111240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eef971-10d5-4c62-923f-5753d29f6c33 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:45.125695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 233/466] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:45.127926 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95dce69-21f3-4bf6-a451-c4f99e333c78 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:45.131222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eef971-10d5-4c62-923f-5753d29f6c33 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:45.268062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eef971-10d5-4c62-923f-5753d29f6c33 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:45.273727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:45.289072 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95dce69-21f3-4bf6-a451-c4f99e333c78 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:45.311332 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af7272-002a-4f1a-9c6e-a42fcfa208d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:45.312331 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af7272-002a-4f1a-9c6e-a42fcfa208d2 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.312813 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af7272-002a-4f1a-9c6e-a42fcfa208d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.313399 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af7272-002a-4f1a-9c6e-a42fcfa208d2 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.314122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98af7272-002a-4f1a-9c6e-a42fcfa208d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:45.315572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98af7272-002a-4f1a-9c6e-a42fcfa208d2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:45.339381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7570c-b3f0-4512-a909-7f152f3b4c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:45.340944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7570c-b3f0-4512-a909-7f152f3b4c40 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.341227 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7570c-b3f0-4512-a909-7f152f3b4c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.341499 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7570c-b3f0-4512-a909-7f152f3b4c40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.348580 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 234/467] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:10:45 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:10:45.349688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5996de-d788-4a71-9a7e-9debdbbb916c None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.349983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5996de-d788-4a71-9a7e-9debdbbb916c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.350268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5996de-d788-4a71-9a7e-9debdbbb916c None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.352591 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 235/468] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:45.357370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e790d43-c956-4a40-8109-655a10a9ae3f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.358016 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e790d43-c956-4a40-8109-655a10a9ae3f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.358675 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e790d43-c956-4a40-8109-655a10a9ae3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.360693 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7570c-b3f0-4512-a909-7f152f3b4c40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:45.362171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7570c-b3f0-4512-a909-7f152f3b4c40 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:45.378960 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5e790d43-c956-4a40-8109-655a10a9ae3f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:45.380601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5e790d43-c956-4a40-8109-655a10a9ae3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:45.382084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-e5e7570c-b3f0-4512-a909-7f152f3b4c40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:45.382084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:45.382084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:45.382084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:45.382084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:45.382084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:45.382084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:45.382084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:45.382084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:45.382084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:45.382084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:45.382084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:45.382084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:45.382084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:45.382084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:45.382878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:45.383732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:45.383732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:45.383732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:45.383732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:45.388253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 234/469] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:45.389540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7570c-b3f0-4512-a909-7f152f3b4c40 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:45.436189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:45.464977 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 236/470] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:45.489251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e790d43-c956-4a40-8109-655a10a9ae3f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:45.504861 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7570c-b3f0-4512-a909-7f152f3b4c40 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:45.551726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37432c-b847-4d7a-9ff6-33d397f51e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:45.552975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37432c-b847-4d7a-9ff6-33d397f51e00 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.553263 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37432c-b847-4d7a-9ff6-33d397f51e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.553544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37432c-b847-4d7a-9ff6-33d397f51e00 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.554300 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37432c-b847-4d7a-9ff6-33d397f51e00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:45.554610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37432c-b847-4d7a-9ff6-33d397f51e00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:45.556025 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37432c-b847-4d7a-9ff6-33d397f51e00 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:45.577357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:45.582473 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 235/471] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:45.588182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37432c-b847-4d7a-9ff6-33d397f51e00 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:45.653043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e790d43-c956-4a40-8109-655a10a9ae3f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:45.704320 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22417d-5809-4d2b-9e3f-293b3ee6afb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:45.704320 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22417d-5809-4d2b-9e3f-293b3ee6afb8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.704320 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22417d-5809-4d2b-9e3f-293b3ee6afb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.704320 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22417d-5809-4d2b-9e3f-293b3ee6afb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.714959 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37432c-b847-4d7a-9ff6-33d397f51e00 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:45.718698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22417d-5809-4d2b-9e3f-293b3ee6afb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:45.721296 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22417d-5809-4d2b-9e3f-293b3ee6afb8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:45.746917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-2d22417d-5809-4d2b-9e3f-293b3ee6afb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:45.746917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:45.746917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:45.746917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:45.746917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:45.746917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:45.746917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:45.746917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:45.746917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:45.746917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:45.746917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:45.746917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:45.746917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:45.746917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:45.746917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:45.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:45.748876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:45.748876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:45.748876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:45.748876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:45.749085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 237/472] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:45.754891 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22417d-5809-4d2b-9e3f-293b3ee6afb8 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:45.766689 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c9273-3d10-4d75-9ff0-20428a3ed213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:45.767741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c9273-3d10-4d75-9ff0-20428a3ed213 None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.768162 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c9273-3d10-4d75-9ff0-20428a3ed213 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.768595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c9273-3d10-4d75-9ff0-20428a3ed213 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.769463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c9273-3d10-4d75-9ff0-20428a3ed213 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:45.771292 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c9273-3d10-4d75-9ff0-20428a3ed213 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:45.796849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:45.797473 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 236/473] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:10:45 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:45.841045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55e69e-f4e8-46fa-a708-c44daba47242 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.841251 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55e69e-f4e8-46fa-a708-c44daba47242 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.841552 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55e69e-f4e8-46fa-a708-c44daba47242 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.847320 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 237/474] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:45.849854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31253acc-eb95-44fc-af39-b37e0b2d1be4 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.850155 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31253acc-eb95-44fc-af39-b37e0b2d1be4 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.850469 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31253acc-eb95-44fc-af39-b37e0b2d1be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.870059 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-31253acc-eb95-44fc-af39-b37e0b2d1be4 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:45.871314 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-31253acc-eb95-44fc-af39-b37e0b2d1be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:45.879151 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22417d-5809-4d2b-9e3f-293b3ee6afb8 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:45.925982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2578ea-ab84-44d3-9ab9-f3421b8ea1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:45.926912 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2578ea-ab84-44d3-9ab9-f3421b8ea1b4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.927220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2578ea-ab84-44d3-9ab9-f3421b8ea1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.928048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2578ea-ab84-44d3-9ab9-f3421b8ea1b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.929115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2578ea-ab84-44d3-9ab9-f3421b8ea1b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:45.929428 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2578ea-ab84-44d3-9ab9-f3421b8ea1b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:45.931501 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:45.933037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2578ea-ab84-44d3-9ab9-f3421b8ea1b4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:45.949476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 238/475] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:45.957503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2578ea-ab84-44d3-9ab9-f3421b8ea1b4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:45.979316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 238/476] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:46.026355 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31253acc-eb95-44fc-af39-b37e0b2d1be4 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:46.095224 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:46.106801 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2578ea-ab84-44d3-9ab9-f3421b8ea1b4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:46.143723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c80cd-39a5-49af-9188-8166d77c16fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:46.144685 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c80cd-39a5-49af-9188-8166d77c16fb None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:46.145112 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c80cd-39a5-49af-9188-8166d77c16fb None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:46.145592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c80cd-39a5-49af-9188-8166d77c16fb None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:46.146978 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c80cd-39a5-49af-9188-8166d77c16fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:46.148630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c80cd-39a5-49af-9188-8166d77c16fb None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:46.152650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31253acc-eb95-44fc-af39-b37e0b2d1be4 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:46.172198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 239/477] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:10:45 2022] POST /identity/v3/users => generated 249 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:46.196948 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9fc17-ebd9-4191-9dea-07b20f518fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:46.197895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9fc17-ebd9-4191-9dea-07b20f518fc5 None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:46.198323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9fc17-ebd9-4191-9dea-07b20f518fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:46.198766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9fc17-ebd9-4191-9dea-07b20f518fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:46.199745 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9fc17-ebd9-4191-9dea-07b20f518fc5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:46.201450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9fc17-ebd9-4191-9dea-07b20f518fc5 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:46.211356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.sql.core [None req-bdb9fc17-ebd9-4191-9dea-07b20f518fc5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 19:10:46.211356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 19:10:46.211356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [parameters: {'id': '152ccb8630734be1836e0757045679b3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 19:10:46.211356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80242) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 19:10:46.229639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-bdb9fc17-ebd9-4191-9dea-07b20f518fc5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 19:10:46.229639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:46.229639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:10:46.229639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:10:46.229639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:10:46.229639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:10:46.229639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:10:46.229639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:10:46.229639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:10:46.229639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:10:46.229639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:10:46.229639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:10:46.229639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:10:46.229639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application result.read() Jul 20 19:10:46.229639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return ref.to_dict() Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:10:46.230741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application yield resource Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise self.value Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application session.commit() Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 19:10:46.231750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self.session.flush() Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application compat.raise_( Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 19:10:46.232650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 19:10:46.233829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 19:10:46.233829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 19:10:46.233829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 19:10:46.233829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 19:10:46.233829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 19:10:46.233829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 19:10:46.233829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 19:10:46.233829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 19:10:46.233829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 19:10:46.233829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 19:10:46.233829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 19:10:46.233829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 19:10:46.233829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 19:10:46.233829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 19:10:46.233829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:10:46.233829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception Jul 20 19:10:46.234634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:10:46.234634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:10:46.234634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:10:46.234634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:10:46.234634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:10:46.234634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:10:46.234634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:10:46.234634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:10:46.234634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:10:46.234634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:10:46.234634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:10:46.234634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application result.read() Jul 20 19:10:46.234634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:10:46.234634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:10:46.234634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:10:46.234634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:10:46.235417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:10:46.235417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:10:46.235417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:10:46.235417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:10:46.235417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 19:10:46.235417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 19:10:46.235417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [parameters: {'id': '152ccb8630734be1836e0757045679b3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 19:10:46.235417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 19:10:46.235417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:46.235417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:10:46.235417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:46.235417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:46.235417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:46.235417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:46.235417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:46.235417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:46.236494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:46.236494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:46.236494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:46.236494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:46.236494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:46.236494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:46.236494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 20 19:10:46.236494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 20 19:10:46.236494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:46.236494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:46.236494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 20 19:10:46.236494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 20 19:10:46.236494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 19:10:46.236494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 19:10:46.236494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 19:10:46.236494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:46.239032 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 239/478] 10.222.0.43 () {60 vars in 1131 bytes} [Wed Jul 20 19:10:46 2022] POST /identity/v3/roles => generated 144 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:46.246191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c80cd-39a5-49af-9188-8166d77c16fb None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:46.378587 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c80cd-39a5-49af-9188-8166d77c16fb None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:46.424834 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae5bd4-889b-48d6-8964-e6006a5ca863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:46.425701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae5bd4-889b-48d6-8964-e6006a5ca863 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:46.425991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae5bd4-889b-48d6-8964-e6006a5ca863 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:46.426269 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae5bd4-889b-48d6-8964-e6006a5ca863 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:46.444587 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae5bd4-889b-48d6-8964-e6006a5ca863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:46.453097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-cbae5bd4-889b-48d6-8964-e6006a5ca863 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:46.453097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:46.453097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:46.453097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:46.453097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:46.453097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:46.453097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:46.453097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:46.453097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:46.453097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:46.453097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:46.453097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:46.453097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:46.453097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:46.453097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:46.453097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:46.454041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:46.454041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:46.454041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:46.454041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:46.454041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:46.454041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:46.454041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:46.454041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:46.454041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:46.454041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:46.454041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:46.454041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:46.454041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:46.454041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:46.454041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:46.454041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:46.455312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:46.455312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:46.455312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:46.455312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:46.455312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:46.455312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:46.455312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:46.455312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:46.455312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:46.455312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:46.455312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:46.455312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:46.455312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 240/479] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:46.460251 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9fc17-ebd9-4191-9dea-07b20f518fc5 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:46.520457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:46.584101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9fc17-ebd9-4191-9dea-07b20f518fc5 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:46.620402 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fe85b-329c-4219-94c0-ebed56c94a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:46.621021 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fe85b-329c-4219-94c0-ebed56c94a28 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:46.621302 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fe85b-329c-4219-94c0-ebed56c94a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:46.621575 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fe85b-329c-4219-94c0-ebed56c94a28 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:46.622328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2fe85b-329c-4219-94c0-ebed56c94a28 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:46.622633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2fe85b-329c-4219-94c0-ebed56c94a28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:46.623981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2fe85b-329c-4219-94c0-ebed56c94a28 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:46.633340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 240/480] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:47.458361 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95233d-0e19-464a-8663-061df59ca7a1 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:47.458939 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95233d-0e19-464a-8663-061df59ca7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:47.458939 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95233d-0e19-464a-8663-061df59ca7a1 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:47.460344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 241/481] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:47.465912 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731d279-cb1a-4f80-a2d3-5c2d3d788951 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:47.466324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731d279-cb1a-4f80-a2d3-5c2d3d788951 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:47.466469 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731d279-cb1a-4f80-a2d3-5c2d3d788951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:47.488446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-f731d279-cb1a-4f80-a2d3-5c2d3d788951 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:47.489650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f731d279-cb1a-4f80-a2d3-5c2d3d788951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:47.586710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 241/482] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:47.610212 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95233d-0e19-464a-8663-061df59ca7a1 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:47.619686 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:47.717573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95233d-0e19-464a-8663-061df59ca7a1 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:47.751772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb57c9-bbf2-443d-bbc1-cdde5a16c7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:47.752606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb57c9-bbf2-443d-bbc1-cdde5a16c7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:47.752891 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb57c9-bbf2-443d-bbc1-cdde5a16c7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:47.753166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb57c9-bbf2-443d-bbc1-cdde5a16c7c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:47.763463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb57c9-bbf2-443d-bbc1-cdde5a16c7c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:47.771633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-ffeb57c9-bbf2-443d-bbc1-cdde5a16c7c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:47.771633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:47.771633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:47.771633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:47.771633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:47.771633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:47.771633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:47.771633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:47.771633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:47.771633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:47.771633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:47.771633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:47.771633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:47.771633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:47.771633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:47.771633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:47.772543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:47.772543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:47.772543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:47.772543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:47.772543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:47.772543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:47.772543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:47.772543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:47.772543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:47.772543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:47.772543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:47.772543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:47.772543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:47.772543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:47.772543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:47.772543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:47.773752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:47.773752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:47.773752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:47.773752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:47.773752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:47.773752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:47.773752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:47.773752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:47.773752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:47.773752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:47.773752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:47.773752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:47.773752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 242/483] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:47.778849 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731d279-cb1a-4f80-a2d3-5c2d3d788951 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:47.823830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:47.908401 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731d279-cb1a-4f80-a2d3-5c2d3d788951 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:47.949873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ca518-8177-4b5d-961e-b48869ba409d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:47.950562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ca518-8177-4b5d-961e-b48869ba409d None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:47.950841 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ca518-8177-4b5d-961e-b48869ba409d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:47.951114 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ca518-8177-4b5d-961e-b48869ba409d None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:47.951969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ca518-8177-4b5d-961e-b48869ba409d None admin] RBAC: Adding query filter params (name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:47.952272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ca518-8177-4b5d-961e-b48869ba409d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:47.953648 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ca518-8177-4b5d-961e-b48869ba409d None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:47.964945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 242/484] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:47.973397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb57c9-bbf2-443d-bbc1-cdde5a16c7c2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.027721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913023b-c3af-4a92-8e0f-ca05194abc6b None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.028373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913023b-c3af-4a92-8e0f-ca05194abc6b None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.028876 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913023b-c3af-4a92-8e0f-ca05194abc6b None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.031813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 243/485] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:48.036349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1336db-6409-4089-ac4f-0751ab4d8247 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.036992 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1336db-6409-4089-ac4f-0751ab4d8247 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.037466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1336db-6409-4089-ac4f-0751ab4d8247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.055537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-5f1336db-6409-4089-ac4f-0751ab4d8247 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:48.057302 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5f1336db-6409-4089-ac4f-0751ab4d8247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:48.072260 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:48.109309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb57c9-bbf2-443d-bbc1-cdde5a16c7c2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.140401 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 244/486] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:48.143523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c11b60-9488-4e64-9365-965c24adff8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.144350 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c11b60-9488-4e64-9365-965c24adff8f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.144641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c11b60-9488-4e64-9365-965c24adff8f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.144916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c11b60-9488-4e64-9365-965c24adff8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.156280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c11b60-9488-4e64-9365-965c24adff8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:48.157827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c11b60-9488-4e64-9365-965c24adff8f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:48.158362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1336db-6409-4089-ac4f-0751ab4d8247 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.176187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-e4c11b60-9488-4e64-9365-965c24adff8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:48.176187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:48.176187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:48.176187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:48.176187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:48.176187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:48.176187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:48.176187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:48.176187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:48.176187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:48.176187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:48.176187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:48.176187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:48.176187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:48.176187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:48.177085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:48.178926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:48.178926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:48.178926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:48.178926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:48.178926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 243/487] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:48.183718 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c11b60-9488-4e64-9365-965c24adff8f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.214788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:48.273038 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1336db-6409-4089-ac4f-0751ab4d8247 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.311959 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4acb07-b52b-47ea-94d8-0dad9757c79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.313068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4acb07-b52b-47ea-94d8-0dad9757c79f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.313428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4acb07-b52b-47ea-94d8-0dad9757c79f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.313780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4acb07-b52b-47ea-94d8-0dad9757c79f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.325806 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4acb07-b52b-47ea-94d8-0dad9757c79f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:48.334492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-1a4acb07-b52b-47ea-94d8-0dad9757c79f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:48.334492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:48.334492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:48.334492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:48.334492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:48.334492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:48.334492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:48.334492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:48.334492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:48.334492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:48.334492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:48.334492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:48.334492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:48.334492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:48.334492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:48.334492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:48.335384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:48.335384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:48.335384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:48.335384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:48.335384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:48.335384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:48.335384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:48.335384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:48.335384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:48.335384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:48.335384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:48.335384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:48.335384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:48.335384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:48.335384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:48.335384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:48.336144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:48.336144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:48.336144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:48.336144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:48.336144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:48.336144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:48.336144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:48.336144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:48.336144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:48.336144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:48.336144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:48.336144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:48.341342 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 245/488] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:48.343135 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4acb07-b52b-47ea-94d8-0dad9757c79f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.454247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4acb07-b52b-47ea-94d8-0dad9757c79f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.462247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c11b60-9488-4e64-9365-965c24adff8f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.464670 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:48.492589 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf64a6-b5cb-4d8b-b37c-4a42c73d14b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.493330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf64a6-b5cb-4d8b-b37c-4a42c73d14b5 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.493552 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf64a6-b5cb-4d8b-b37c-4a42c73d14b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.493835 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf64a6-b5cb-4d8b-b37c-4a42c73d14b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.494597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf64a6-b5cb-4d8b-b37c-4a42c73d14b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:48.494910 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf64a6-b5cb-4d8b-b37c-4a42c73d14b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:48.496633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf64a6-b5cb-4d8b-b37c-4a42c73d14b5 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:48.511150 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 246/489] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:48.512564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf64a6-b5cb-4d8b-b37c-4a42c73d14b5 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.519577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:48.582911 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd6f8d-b705-4a24-8954-e753968097f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.583628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd6f8d-b705-4a24-8954-e753968097f7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.583901 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd6f8d-b705-4a24-8954-e753968097f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.584180 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd6f8d-b705-4a24-8954-e753968097f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.584950 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd6f8d-b705-4a24-8954-e753968097f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:48.585259 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd6f8d-b705-4a24-8954-e753968097f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:48.586642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd6f8d-b705-4a24-8954-e753968097f7 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:48.622715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 244/490] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:48.627808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd6f8d-b705-4a24-8954-e753968097f7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.673379 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf64a6-b5cb-4d8b-b37c-4a42c73d14b5 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.720783 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80b73d-7736-40ba-b19e-abc36976e39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.721508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80b73d-7736-40ba-b19e-abc36976e39c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.721942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80b73d-7736-40ba-b19e-abc36976e39c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.722226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80b73d-7736-40ba-b19e-abc36976e39c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.735866 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80b73d-7736-40ba-b19e-abc36976e39c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:48.737370 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80b73d-7736-40ba-b19e-abc36976e39c None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:48.757597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-dc80b73d-7736-40ba-b19e-abc36976e39c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:48.757597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:48.757597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:48.757597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:48.757597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:48.757597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:48.757597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:48.757597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:48.757597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:48.757597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:48.757597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:48.757597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:48.757597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:48.757597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:48.757597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:48.758492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:48.759319 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:48.759319 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:48.759319 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:48.759319 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:48.759319 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 247/491] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:48.764543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946642f-f301-403c-9891-27bb6f80b290 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.764769 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946642f-f301-403c-9891-27bb6f80b290 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.765050 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946642f-f301-403c-9891-27bb6f80b290 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.766895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 248/492] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:48.767598 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4946642f-f301-403c-9891-27bb6f80b290 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.777286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd6f8d-b705-4a24-8954-e753968097f7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.778483 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:48.821443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37490a-7c42-40d0-9085-56b42138b381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.822387 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37490a-7c42-40d0-9085-56b42138b381 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.822674 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37490a-7c42-40d0-9085-56b42138b381 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.822949 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37490a-7c42-40d0-9085-56b42138b381 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.832123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37490a-7c42-40d0-9085-56b42138b381 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:48.834265 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37490a-7c42-40d0-9085-56b42138b381 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:48.854727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-4d37490a-7c42-40d0-9085-56b42138b381 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:10:48.854727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:48.854727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:48.854727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:48.854727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:48.854727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:48.854727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:48.854727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:48.854727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:48.854727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:48.854727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:48.854727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:48.854727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:48.854727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:48.854727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:48.854727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:48.855705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:48.857065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:48.857065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:48.857065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:48.857065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:48.857065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:48.857065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:48.857065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:48.857065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:48.857065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:10:48.857065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:48.858323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 245/493] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:48.859240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62896b7d-957d-4d3c-ad10-662f99760176 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.859529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62896b7d-957d-4d3c-ad10-662f99760176 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.859810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62896b7d-957d-4d3c-ad10-662f99760176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.876772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-62896b7d-957d-4d3c-ad10-662f99760176 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:48.877951 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-62896b7d-957d-4d3c-ad10-662f99760176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:48.880752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4946642f-f301-403c-9891-27bb6f80b290 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.908859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:48.917009 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d821d8-e629-4be4-a90f-4878360bb7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.917673 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d821d8-e629-4be4-a90f-4878360bb7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.917964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d821d8-e629-4be4-a90f-4878360bb7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.918241 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d821d8-e629-4be4-a90f-4878360bb7a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.918969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d821d8-e629-4be4-a90f-4878360bb7a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:48.919276 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d821d8-e629-4be4-a90f-4878360bb7a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:48.920682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d821d8-e629-4be4-a90f-4878360bb7a8 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:48.938143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 249/494] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:48.938143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009d68b-4b7f-4e32-a2c8-6a510162043a None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.938143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009d68b-4b7f-4e32-a2c8-6a510162043a None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.938143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009d68b-4b7f-4e32-a2c8-6a510162043a None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.938143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 250/495] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:48.938143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009d68b-4b7f-4e32-a2c8-6a510162043a None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.954802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 246/496] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:48.957560 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62896b7d-957d-4d3c-ad10-662f99760176 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:49.014153 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:49.055260 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009d68b-4b7f-4e32-a2c8-6a510162043a None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.077672 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62896b7d-957d-4d3c-ad10-662f99760176 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.091814 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221b80e-51d6-4115-bdf3-ddf23018a6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.092526 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221b80e-51d6-4115-bdf3-ddf23018a6da None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.092815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221b80e-51d6-4115-bdf3-ddf23018a6da None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.093100 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221b80e-51d6-4115-bdf3-ddf23018a6da None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.093952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221b80e-51d6-4115-bdf3-ddf23018a6da None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:49.094267 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221b80e-51d6-4115-bdf3-ddf23018a6da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:49.095707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221b80e-51d6-4115-bdf3-ddf23018a6da None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:49.113708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dadba9-642b-447d-9098-05f3b16e8402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.114505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dadba9-642b-447d-9098-05f3b16e8402 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.115318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dadba9-642b-447d-9098-05f3b16e8402 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.115612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dadba9-642b-447d-9098-05f3b16e8402 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.123449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dadba9-642b-447d-9098-05f3b16e8402 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:49.124930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dadba9-642b-447d-9098-05f3b16e8402 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:49.139485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 251/497] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:49.139485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12ad4e-6d6b-4c5a-8e48-b77972d14fb2 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.139485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12ad4e-6d6b-4c5a-8e48-b77972d14fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.139485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12ad4e-6d6b-4c5a-8e48-b77972d14fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.140191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-78dadba9-642b-447d-9098-05f3b16e8402 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:10:49.140191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:49.140191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:49.140191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:49.140191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:49.140191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:49.140191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:49.140191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:49.140191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:49.140191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:49.140191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:49.140191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:49.140191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:49.140191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:49.140191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:49.140191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:49.141055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:49.141914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:49.141914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:49.141914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:49.141914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:49.141914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:49.141914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:49.141914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:49.141914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:49.141914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:10:49.141914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:49.141914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 247/498] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:49.142456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dadba9-642b-447d-9098-05f3b16e8402 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:49.158037 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-ca12ad4e-6d6b-4c5a-8e48-b77972d14fb2 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:49.159243 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ca12ad4e-6d6b-4c5a-8e48-b77972d14fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:49.178057 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:49.237463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 252/499] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:49.238364 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12ad4e-6d6b-4c5a-8e48-b77972d14fb2 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:49.271614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dadba9-642b-447d-9098-05f3b16e8402 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.296895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:49.307655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd87637-eee2-41a0-a8f3-5c0c7242c89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.308843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd87637-eee2-41a0-a8f3-5c0c7242c89a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.309221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd87637-eee2-41a0-a8f3-5c0c7242c89a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.309594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd87637-eee2-41a0-a8f3-5c0c7242c89a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.320458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd87637-eee2-41a0-a8f3-5c0c7242c89a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:49.328189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-2fd87637-eee2-41a0-a8f3-5c0c7242c89a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:49.328189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:49.328189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:49.328189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:49.328189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:49.328189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:49.328189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:49.328189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:49.328189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:49.328189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:49.328189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:49.328189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:49.328189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:49.328189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:49.328189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:49.328189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:49.329100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:49.329100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:49.329100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:49.329100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:49.329100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:49.329100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:49.329100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:49.329100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:49.329100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:49.329100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:49.329100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:49.329100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:49.329100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:49.329100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:49.329100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:49.329100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:49.329968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:49.329968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:49.329968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:49.329968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:49.329968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:49.329968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:49.329968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:49.329968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:49.329968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:49.329968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:49.329968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:49.329968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:49.334409 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 248/500] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:49.335291 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd87637-eee2-41a0-a8f3-5c0c7242c89a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:49.354071 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12ad4e-6d6b-4c5a-8e48-b77972d14fb2 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.394845 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf151f5e-74e2-4e93-9245-d16ef86fced3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.395759 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf151f5e-74e2-4e93-9245-d16ef86fced3 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.396192 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf151f5e-74e2-4e93-9245-d16ef86fced3 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.396623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf151f5e-74e2-4e93-9245-d16ef86fced3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.435362 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:49.448738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd87637-eee2-41a0-a8f3-5c0c7242c89a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.469897 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf151f5e-74e2-4e93-9245-d16ef86fced3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:49.471636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf151f5e-74e2-4e93-9245-d16ef86fced3 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:49.481904 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b8a66-96fd-42fd-8bd9-163576f01414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.482574 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b8a66-96fd-42fd-8bd9-163576f01414 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.482860 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b8a66-96fd-42fd-8bd9-163576f01414 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.483136 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b8a66-96fd-42fd-8bd9-163576f01414 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.483985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b8a66-96fd-42fd-8bd9-163576f01414 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:49.484318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b8a66-96fd-42fd-8bd9-163576f01414 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:49.485774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b8a66-96fd-42fd-8bd9-163576f01414 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:49.523434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 249/501] 10.222.0.43 () {56 vars in 1137 bytes} [Wed Jul 20 19:10:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:49.524188 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b8a66-96fd-42fd-8bd9-163576f01414 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:49.528259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:49.572884 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 253/502] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:49.573742 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf151f5e-74e2-4e93-9245-d16ef86fced3 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:49.626255 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b8a66-96fd-42fd-8bd9-163576f01414 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.660000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb0cf0-b5a7-4448-b3c6-2dc19c92899a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.660826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb0cf0-b5a7-4448-b3c6-2dc19c92899a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.661105 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb0cf0-b5a7-4448-b3c6-2dc19c92899a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.661379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb0cf0-b5a7-4448-b3c6-2dc19c92899a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.672944 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb0cf0-b5a7-4448-b3c6-2dc19c92899a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:49.674608 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf151f5e-74e2-4e93-9245-d16ef86fced3 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.680644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-c3bb0cf0-b5a7-4448-b3c6-2dc19c92899a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:49.680644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:49.680644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:49.680644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:49.680644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:49.680644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:49.680644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:49.680644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:49.680644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:49.680644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:49.680644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:49.680644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:49.680644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:49.680644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:49.680644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:49.680644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:49.681500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:49.681500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:49.681500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:49.681500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:49.681500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:49.681500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:49.681500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:49.681500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:49.681500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:49.681500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:49.681500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:49.681500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:49.681500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:49.681500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:49.681500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:49.681500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:49.687744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:49.687744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:49.687744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:49.687744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:49.687744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:49.687744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:49.687744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:49.687744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:49.687744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:49.687744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:49.687744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:49.687744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:49.687744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 250/503] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:49.687744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc899778-15e8-4fff-b4a9-5d62ed87d67b None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.688537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc899778-15e8-4fff-b4a9-5d62ed87d67b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.688537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc899778-15e8-4fff-b4a9-5d62ed87d67b None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.688537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 251/504] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:49.688537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc899778-15e8-4fff-b4a9-5d62ed87d67b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:49.695551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:49.713209 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1a92e-53b6-4399-976b-c85f4916b04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.713874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1a92e-53b6-4399-976b-c85f4916b04d None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.714154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1a92e-53b6-4399-976b-c85f4916b04d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.714422 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1a92e-53b6-4399-976b-c85f4916b04d None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.715154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd1a92e-53b6-4399-976b-c85f4916b04d None admin] RBAC: Adding query filter params (name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:49.715457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd1a92e-53b6-4399-976b-c85f4916b04d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:49.717052 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd1a92e-53b6-4399-976b-c85f4916b04d None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:49.724838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 254/505] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:49.725634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1a92e-53b6-4399-976b-c85f4916b04d None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:49.743801 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:49.791523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc899778-15e8-4fff-b4a9-5d62ed87d67b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.824112 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1a92e-53b6-4399-976b-c85f4916b04d None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.825147 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c0f03-2008-4cad-94cb-fc714f5de7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.825788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c0f03-2008-4cad-94cb-fc714f5de7cd None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.826067 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c0f03-2008-4cad-94cb-fc714f5de7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.826349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c0f03-2008-4cad-94cb-fc714f5de7cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.856754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f188e-f86e-416e-a254-9a9e16317a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.857537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f188e-f86e-416e-a254-9a9e16317a2e None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.857819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f188e-f86e-416e-a254-9a9e16317a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.858091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f188e-f86e-416e-a254-9a9e16317a2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.869696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f188e-f86e-416e-a254-9a9e16317a2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:49.871159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f188e-f86e-416e-a254-9a9e16317a2e None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:49.888653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-eb2f188e-f86e-416e-a254-9a9e16317a2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:49.888653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:49.888653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:49.888653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:49.888653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:49.888653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:49.888653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:49.888653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:49.888653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:49.888653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:49.888653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:49.888653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:49.888653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:49.888653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:49.888653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:49.889500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:49.890328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:49.890328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:49.890328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:49.890328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:49.890328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 255/506] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:49.890591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f188e-f86e-416e-a254-9a9e16317a2e None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:49.896770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c0f03-2008-4cad-94cb-fc714f5de7cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:49.898310 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c0f03-2008-4cad-94cb-fc714f5de7cd None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:49.906059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:49.961614 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:49.992828 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f188e-f86e-416e-a254-9a9e16317a2e None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.997270 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 252/507] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:49.998581 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db08093-afde-4a93-98d1-5e8c1948b99c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.998865 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db08093-afde-4a93-98d1-5e8c1948b99c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.999142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db08093-afde-4a93-98d1-5e8c1948b99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.015301 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7db08093-afde-4a93-98d1-5e8c1948b99c None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:50.016518 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7db08093-afde-4a93-98d1-5e8c1948b99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:50.026584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f71f1-35ee-44d8-8df7-f18018c4c14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.027255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f71f1-35ee-44d8-8df7-f18018c4c14b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.027539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f71f1-35ee-44d8-8df7-f18018c4c14b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.027817 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f71f1-35ee-44d8-8df7-f18018c4c14b None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.028599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f71f1-35ee-44d8-8df7-f18018c4c14b None admin] RBAC: Adding query filter params (name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:50.028909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f71f1-35ee-44d8-8df7-f18018c4c14b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.030259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f71f1-35ee-44d8-8df7-f18018c4c14b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.038019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 256/508] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:50.038795 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f71f1-35ee-44d8-8df7-f18018c4c14b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.082293 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 253/509] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:50.083238 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db08093-afde-4a93-98d1-5e8c1948b99c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.129032 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:50.140377 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f71f1-35ee-44d8-8df7-f18018c4c14b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.168538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:50.173933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e9e63-3746-4e38-9e4f-ebaf3bca1959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.174721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e9e63-3746-4e38-9e4f-ebaf3bca1959 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.175007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e9e63-3746-4e38-9e4f-ebaf3bca1959 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.175289 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e9e63-3746-4e38-9e4f-ebaf3bca1959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.185974 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e9e63-3746-4e38-9e4f-ebaf3bca1959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.187421 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e9e63-3746-4e38-9e4f-ebaf3bca1959 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.194919 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db08093-afde-4a93-98d1-5e8c1948b99c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.204718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-156e9e63-3746-4e38-9e4f-ebaf3bca1959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:50.204718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:50.204718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:50.204718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:50.204718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:50.204718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:50.204718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:50.204718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:50.204718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:50.204718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:50.204718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:50.204718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:50.204718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:50.204718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:50.204718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:50.205539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:50.206380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:50.206380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:50.206380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:50.206380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:50.206380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 257/510] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:50.206725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e9e63-3746-4e38-9e4f-ebaf3bca1959 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.237912 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126fae1-99f2-4b07-918c-b0eed1e2fb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.238665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126fae1-99f2-4b07-918c-b0eed1e2fb87 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.239015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126fae1-99f2-4b07-918c-b0eed1e2fb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.239295 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126fae1-99f2-4b07-918c-b0eed1e2fb87 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.240054 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126fae1-99f2-4b07-918c-b0eed1e2fb87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:50.240390 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126fae1-99f2-4b07-918c-b0eed1e2fb87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.241760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126fae1-99f2-4b07-918c-b0eed1e2fb87 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.254028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 254/511] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:50.254857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126fae1-99f2-4b07-918c-b0eed1e2fb87 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.313409 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e9e63-3746-4e38-9e4f-ebaf3bca1959 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.350589 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb70dd2-f1e1-461c-a99e-46470e6ffa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.351262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb70dd2-f1e1-461c-a99e-46470e6ffa9d None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.351547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb70dd2-f1e1-461c-a99e-46470e6ffa9d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.351825 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb70dd2-f1e1-461c-a99e-46470e6ffa9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.353031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:50.360423 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126fae1-99f2-4b07-918c-b0eed1e2fb87 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.362575 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb70dd2-f1e1-461c-a99e-46470e6ffa9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.364035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb70dd2-f1e1-461c-a99e-46470e6ffa9d None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.381273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-ebb70dd2-f1e1-461c-a99e-46470e6ffa9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:50.381273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:50.381273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:50.381273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:50.381273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:50.381273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:50.381273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:50.381273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:50.381273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:50.381273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:50.381273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:50.381273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:50.381273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:50.381273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:50.381273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:50.382124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:50.382952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:50.382952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:50.382952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:50.382952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:50.382952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 258/512] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:50.383531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb70dd2-f1e1-461c-a99e-46470e6ffa9d None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.397378 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c53a7-5549-4ae0-b255-fb5083f6be22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.398176 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c53a7-5549-4ae0-b255-fb5083f6be22 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.398460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c53a7-5549-4ae0-b255-fb5083f6be22 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.398738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c53a7-5549-4ae0-b255-fb5083f6be22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.409476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c53a7-5549-4ae0-b255-fb5083f6be22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.410923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c53a7-5549-4ae0-b255-fb5083f6be22 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.421386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:50.428044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-360c53a7-5549-4ae0-b255-fb5083f6be22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:50.428044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:50.428044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:50.428044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:50.428044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:50.428044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:50.428044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:50.428044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:50.428044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:50.428044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:50.428044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:50.428044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:50.428044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:50.428044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:50.428044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:50.428935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:50.429773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:50.429773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:50.429773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:50.429773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:50.429773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 255/513] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:50.430049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c53a7-5549-4ae0-b255-fb5083f6be22 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.489777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb70dd2-f1e1-461c-a99e-46470e6ffa9d None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.522674 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad66806-f31f-48d1-8d32-c22d69b541bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.523457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad66806-f31f-48d1-8d32-c22d69b541bf None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.523742 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad66806-f31f-48d1-8d32-c22d69b541bf None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.524012 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad66806-f31f-48d1-8d32-c22d69b541bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.531963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c53a7-5549-4ae0-b255-fb5083f6be22 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.534253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad66806-f31f-48d1-8d32-c22d69b541bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.535696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad66806-f31f-48d1-8d32-c22d69b541bf None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.552873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-8ad66806-f31f-48d1-8d32-c22d69b541bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:50.552873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:50.552873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:50.552873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:50.552873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:50.552873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:50.552873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:50.552873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:50.552873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:50.552873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:50.552873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:50.552873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:50.552873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:50.552873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:50.552873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:50.553915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:50.554757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:50.554757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:50.554757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:50.554757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:50.554757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 259/514] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:50.555022 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad66806-f31f-48d1-8d32-c22d69b541bf None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.570258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da910f4b-3a02-4386-b020-d0e890bd0002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.570914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da910f4b-3a02-4386-b020-d0e890bd0002 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.571195 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da910f4b-3a02-4386-b020-d0e890bd0002 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.571467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da910f4b-3a02-4386-b020-d0e890bd0002 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.572187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da910f4b-3a02-4386-b020-d0e890bd0002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:50.572526 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da910f4b-3a02-4386-b020-d0e890bd0002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.573876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da910f4b-3a02-4386-b020-d0e890bd0002 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.585843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 256/515] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:50.586636 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da910f4b-3a02-4386-b020-d0e890bd0002 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.598012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:50.643662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:50.656729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad66806-f31f-48d1-8d32-c22d69b541bf None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.687940 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da910f4b-3a02-4386-b020-d0e890bd0002 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.691451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac26c70-c7a1-4d53-bd83-c55863f978b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.692246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac26c70-c7a1-4d53-bd83-c55863f978b2 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.692564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac26c70-c7a1-4d53-bd83-c55863f978b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.692848 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac26c70-c7a1-4d53-bd83-c55863f978b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.700149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac26c70-c7a1-4d53-bd83-c55863f978b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.701627 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac26c70-c7a1-4d53-bd83-c55863f978b2 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.716578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-1ac26c70-c7a1-4d53-bd83-c55863f978b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:50.716578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:50.716578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:50.716578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:50.716578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:50.716578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:50.716578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:50.716578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:50.716578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:50.716578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:50.716578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:50.716578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:50.716578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:50.716578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:50.716578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:50.717415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:50.718237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:50.718237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:50.718237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:50.718237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:50.718237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:50.718237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:50.718237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:50.718237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:50.718237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 260/516] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:50.718696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac26c70-c7a1-4d53-bd83-c55863f978b2 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.721910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef5847-0a62-4fd3-b930-a643ccd2f19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.722568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef5847-0a62-4fd3-b930-a643ccd2f19f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.722852 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef5847-0a62-4fd3-b930-a643ccd2f19f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.723129 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef5847-0a62-4fd3-b930-a643ccd2f19f None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.723854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef5847-0a62-4fd3-b930-a643ccd2f19f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:50.724194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef5847-0a62-4fd3-b930-a643ccd2f19f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.725582 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef5847-0a62-4fd3-b930-a643ccd2f19f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.736921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 257/517] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:50.737560 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef5847-0a62-4fd3-b930-a643ccd2f19f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.802927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:50.819270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac26c70-c7a1-4d53-bd83-c55863f978b2 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.836564 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef5847-0a62-4fd3-b930-a643ccd2f19f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.852988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5461f6-3ea7-482c-b02a-263b28aa4aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.853680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5461f6-3ea7-482c-b02a-263b28aa4aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.853961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5461f6-3ea7-482c-b02a-263b28aa4aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.854230 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5461f6-3ea7-482c-b02a-263b28aa4aa9 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.854955 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5461f6-3ea7-482c-b02a-263b28aa4aa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:50.855260 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5461f6-3ea7-482c-b02a-263b28aa4aa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.856674 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5461f6-3ea7-482c-b02a-263b28aa4aa9 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.867920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 261/518] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:50.868747 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5461f6-3ea7-482c-b02a-263b28aa4aa9 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.874126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f2e4d-a38d-4866-9e59-5a4e99355855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.874791 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f2e4d-a38d-4866-9e59-5a4e99355855 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.875076 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f2e4d-a38d-4866-9e59-5a4e99355855 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.875353 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f2e4d-a38d-4866-9e59-5a4e99355855 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.876105 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f2e4d-a38d-4866-9e59-5a4e99355855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:50.876445 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f2e4d-a38d-4866-9e59-5a4e99355855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.877818 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f2e4d-a38d-4866-9e59-5a4e99355855 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.882947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:50.891302 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 258/519] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:50.892308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f2e4d-a38d-4866-9e59-5a4e99355855 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.977979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5461f6-3ea7-482c-b02a-263b28aa4aa9 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.989630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f2e4d-a38d-4866-9e59-5a4e99355855 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.011436 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c5a74-75a6-4bcd-a0b4-b40855d1b85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.012236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c5a74-75a6-4bcd-a0b4-b40855d1b85d None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.012548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c5a74-75a6-4bcd-a0b4-b40855d1b85d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.012824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c5a74-75a6-4bcd-a0b4-b40855d1b85d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.015701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:51.020953 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c5a74-75a6-4bcd-a0b4-b40855d1b85d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.022995 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c5a74-75a6-4bcd-a0b4-b40855d1b85d None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.024051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e0fea-6519-4d1f-8ca3-5e3586e2e05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.024737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e0fea-6519-4d1f-8ca3-5e3586e2e05e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.025015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e0fea-6519-4d1f-8ca3-5e3586e2e05e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.025287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e0fea-6519-4d1f-8ca3-5e3586e2e05e None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.026074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e0fea-6519-4d1f-8ca3-5e3586e2e05e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:51.026379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e0fea-6519-4d1f-8ca3-5e3586e2e05e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.027786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e0fea-6519-4d1f-8ca3-5e3586e2e05e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.038386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-840c5a74-75a6-4bcd-a0b4-b40855d1b85d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:10:51.038386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:51.038386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:51.038386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:51.038386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:51.038386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:51.038386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:51.038386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:51.038386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:51.038386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:51.038386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:51.038386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:51.038386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:51.038386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:51.038386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:51.038386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:51.039270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:51.040184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:51.040184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:51.040184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:51.040184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:51.040184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:51.040184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:51.040184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:51.040184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:51.040184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:10:51.040184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:51.040184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 262/520] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:51.041155 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c5a74-75a6-4bcd-a0b4-b40855d1b85d None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.045215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 259/521] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:51.046086 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e0fea-6519-4d1f-8ca3-5e3586e2e05e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.107077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:51.141337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c5a74-75a6-4bcd-a0b4-b40855d1b85d None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.143973 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e0fea-6519-4d1f-8ca3-5e3586e2e05e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.175854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b1097-54cb-4a5d-bdaf-a7b4abb6afd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.176648 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b1097-54cb-4a5d-bdaf-a7b4abb6afd7 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.176927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b1097-54cb-4a5d-bdaf-a7b4abb6afd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.177197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b1097-54cb-4a5d-bdaf-a7b4abb6afd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.178312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd5bfd-753b-4637-865e-1a287832022a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.179106 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd5bfd-753b-4637-865e-1a287832022a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.179388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd5bfd-753b-4637-865e-1a287832022a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.179659 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd5bfd-753b-4637-865e-1a287832022a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.184247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b1097-54cb-4a5d-bdaf-a7b4abb6afd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.185738 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b1097-54cb-4a5d-bdaf-a7b4abb6afd7 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.187123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd5bfd-753b-4637-865e-1a287832022a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.188582 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd5bfd-753b-4637-865e-1a287832022a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.201008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-f99b1097-54cb-4a5d-bdaf-a7b4abb6afd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:51.201008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:51.201008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:51.201008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:51.201008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:51.201008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:51.201008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:51.201008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:51.201008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:51.201008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:51.201008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:51.201008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:51.201008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:51.201008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:51.201008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:51.201909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:51.202727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:51.202727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:51.202727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:51.202727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:51.202727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:51.202727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:51.202727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:51.202727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:51.202727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 263/522] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:51.203190 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b1097-54cb-4a5d-bdaf-a7b4abb6afd7 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.203917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-36cd5bfd-753b-4637-865e-1a287832022a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:10:51.203917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:51.203917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:51.203917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:51.203917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:51.203917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:51.203917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:51.203917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:51.203917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:51.203917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:51.203917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:51.203917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:51.203917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:51.203917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:51.203917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:51.203917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:51.204793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:51.205660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:51.205660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:51.205660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:51.205660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:51.205660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:51.205660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:51.205660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:51.205660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:51.205660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:10:51.205660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:51.205660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 260/523] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:51.206192 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd5bfd-753b-4637-865e-1a287832022a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.257396 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:51.306009 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b1097-54cb-4a5d-bdaf-a7b4abb6afd7 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.309627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd5bfd-753b-4637-865e-1a287832022a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.335311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:51.344312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512543f7-c800-409c-875a-9eead34e55d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.345169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512543f7-c800-409c-875a-9eead34e55d0 None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.345427 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cbd17b-0b4a-493d-9ca1-1e69fe1f7467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.345571 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512543f7-c800-409c-875a-9eead34e55d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.345729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512543f7-c800-409c-875a-9eead34e55d0 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.346096 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cbd17b-0b4a-493d-9ca1-1e69fe1f7467 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.346383 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cbd17b-0b4a-493d-9ca1-1e69fe1f7467 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.346492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512543f7-c800-409c-875a-9eead34e55d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.346664 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cbd17b-0b4a-493d-9ca1-1e69fe1f7467 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.347517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cbd17b-0b4a-493d-9ca1-1e69fe1f7467 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:51.347832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cbd17b-0b4a-493d-9ca1-1e69fe1f7467 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.347925 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512543f7-c800-409c-875a-9eead34e55d0 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.349334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cbd17b-0b4a-493d-9ca1-1e69fe1f7467 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.367370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 261/524] 10.222.0.43 () {56 vars in 1137 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:51.378812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 264/525] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:10:51 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:51.378812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512543f7-c800-409c-875a-9eead34e55d0 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.379111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cbd17b-0b4a-493d-9ca1-1e69fe1f7467 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.484032 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512543f7-c800-409c-875a-9eead34e55d0 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.485168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cbd17b-0b4a-493d-9ca1-1e69fe1f7467 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.494458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:51.524397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f539fd1-f837-4039-a324-7b65b145705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.525002 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f539fd1-f837-4039-a324-7b65b145705b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.525286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f539fd1-f837-4039-a324-7b65b145705b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.525564 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f539fd1-f837-4039-a324-7b65b145705b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.526404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f539fd1-f837-4039-a324-7b65b145705b None admin] RBAC: Adding query filter params (role.id=7fe62c5c771d41c28f790fade1df9357, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=267b178236e046e4bc694b2da6f86eb0) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:51.526715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f539fd1-f837-4039-a324-7b65b145705b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.527080 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b549f-1378-4b4f-817c-7f1fd4523af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.527737 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b549f-1378-4b4f-817c-7f1fd4523af6 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.528017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b549f-1378-4b4f-817c-7f1fd4523af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.528169 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f539fd1-f837-4039-a324-7b65b145705b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.528312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b549f-1378-4b4f-817c-7f1fd4523af6 None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.529104 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b549f-1378-4b4f-817c-7f1fd4523af6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:51.529701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b549f-1378-4b4f-817c-7f1fd4523af6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.531053 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b549f-1378-4b4f-817c-7f1fd4523af6 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.535988 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 262/526] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/role_assignments?user.id=267b178236e046e4bc694b2da6f86eb0&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=7fe62c5c771d41c28f790fade1df9357 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:51.537409 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f539fd1-f837-4039-a324-7b65b145705b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.555821 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 265/527] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:51.558484 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b549f-1378-4b4f-817c-7f1fd4523af6 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.620917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:51.669112 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f539fd1-f837-4039-a324-7b65b145705b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.700507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b549f-1378-4b4f-817c-7f1fd4523af6 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.711118 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592fc0e-b617-4fb5-8dd8-a3aa2307f15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.711803 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592fc0e-b617-4fb5-8dd8-a3aa2307f15c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.712088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592fc0e-b617-4fb5-8dd8-a3aa2307f15c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.712444 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592fc0e-b617-4fb5-8dd8-a3aa2307f15c None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.713246 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592fc0e-b617-4fb5-8dd8-a3aa2307f15c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:51.713558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592fc0e-b617-4fb5-8dd8-a3aa2307f15c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.714994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592fc0e-b617-4fb5-8dd8-a3aa2307f15c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.734963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 263/528] 10.222.0.43 () {56 vars in 1139 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:51.735835 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592fc0e-b617-4fb5-8dd8-a3aa2307f15c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.737605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38d13f-3bc2-46e0-a082-93f69cbc0e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.738331 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38d13f-3bc2-46e0-a082-93f69cbc0e59 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.738899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38d13f-3bc2-46e0-a082-93f69cbc0e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.739335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38d13f-3bc2-46e0-a082-93f69cbc0e59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.768159 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:51.831656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38d13f-3bc2-46e0-a082-93f69cbc0e59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.834025 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38d13f-3bc2-46e0-a082-93f69cbc0e59 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.863239 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592fc0e-b617-4fb5-8dd8-a3aa2307f15c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.888221 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:51.913404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6425253-0764-4f89-a698-072dbe6463f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.914189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6425253-0764-4f89-a698-072dbe6463f1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.914691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6425253-0764-4f89-a698-072dbe6463f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.915037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6425253-0764-4f89-a698-072dbe6463f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.916771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6425253-0764-4f89-a698-072dbe6463f1 None admin] RBAC: Adding query filter params (role.id=7fe62c5c771d41c28f790fade1df9357, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=d3e28557896e41c893f0de2590bc96d7) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:51.917160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6425253-0764-4f89-a698-072dbe6463f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.918684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6425253-0764-4f89-a698-072dbe6463f1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.926836 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 264/529] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/role_assignments?user.id=d3e28557896e41c893f0de2590bc96d7&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=7fe62c5c771d41c28f790fade1df9357 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:51.929334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6425253-0764-4f89-a698-072dbe6463f1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.956316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 266/530] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:51.958134 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38d13f-3bc2-46e0-a082-93f69cbc0e59 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:52.073811 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:52.084504 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38d13f-3bc2-46e0-a082-93f69cbc0e59 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:52.087871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6425253-0764-4f89-a698-072dbe6463f1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:52.134984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e060d-abdc-4f2b-9e67-672d2ae6e53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:52.135636 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e060d-abdc-4f2b-9e67-672d2ae6e53f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.135918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e060d-abdc-4f2b-9e67-672d2ae6e53f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.136191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e060d-abdc-4f2b-9e67-672d2ae6e53f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.156677 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36cb4a-ad89-4269-af80-0709cdf204b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:52.157492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36cb4a-ad89-4269-af80-0709cdf204b9 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.157777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36cb4a-ad89-4269-af80-0709cdf204b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.158054 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36cb4a-ad89-4269-af80-0709cdf204b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.181482 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36cb4a-ad89-4269-af80-0709cdf204b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:52.182964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36cb4a-ad89-4269-af80-0709cdf204b9 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:52.186827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:52.209513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e060d-abdc-4f2b-9e67-672d2ae6e53f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:52.210969 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e060d-abdc-4f2b-9e67-672d2ae6e53f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:52.211712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-fe36cb4a-ad89-4269-af80-0709cdf204b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:52.211712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:52.211712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:52.211712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:52.211712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:52.211712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:52.211712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:52.211712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:52.211712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:52.211712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:52.211712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:52.211712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:52.211712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:52.211712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:52.211712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:52.212568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:52.213396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:52.213396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:52.213396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:52.213396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:52.225384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 267/531] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:52.233241 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36cb4a-ad89-4269-af80-0709cdf204b9 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:52.304667 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:52.314280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 265/532] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:52.319909 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e060d-abdc-4f2b-9e67-672d2ae6e53f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:52.423277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e060d-abdc-4f2b-9e67-672d2ae6e53f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:52.436593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36cb4a-ad89-4269-af80-0709cdf204b9 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:52.459781 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2cee8-ca85-4be0-8233-f661834a8e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:52.460626 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2cee8-ca85-4be0-8233-f661834a8e34 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.460906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2cee8-ca85-4be0-8233-f661834a8e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.461183 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2cee8-ca85-4be0-8233-f661834a8e34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.471895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2cee8-ca85-4be0-8233-f661834a8e34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:52.473481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2cee8-ca85-4be0-8233-f661834a8e34 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:52.480751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5676aa-c1d9-4c16-bf73-c1add0752fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:52.481417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5676aa-c1d9-4c16-bf73-c1add0752fab None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.481706 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5676aa-c1d9-4c16-bf73-c1add0752fab None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.481985 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5676aa-c1d9-4c16-bf73-c1add0752fab None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.482720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5676aa-c1d9-4c16-bf73-c1add0752fab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:52.483028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5676aa-c1d9-4c16-bf73-c1add0752fab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:52.484418 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5676aa-c1d9-4c16-bf73-c1add0752fab None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:52.492684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-34d2cee8-ca85-4be0-8233-f661834a8e34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:52.492684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:52.492684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:52.492684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:52.492684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:52.492684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:52.492684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:52.492684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:52.492684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:52.492684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:52.492684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:52.492684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:52.492684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:52.492684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:52.492684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:52.494628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:52.496740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:52.496740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:52.496740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:52.496740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:52.496740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 266/533] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:52.497383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 268/534] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:52.500446 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2cee8-ca85-4be0-8233-f661834a8e34 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:52.501522 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5676aa-c1d9-4c16-bf73-c1add0752fab None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:52.576644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:52.617045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:52.650616 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2cee8-ca85-4be0-8233-f661834a8e34 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:52.686763 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bd7eb-d0d5-4175-a12b-a934902cef20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:52.688927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bd7eb-d0d5-4175-a12b-a934902cef20 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.689215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bd7eb-d0d5-4175-a12b-a934902cef20 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.689491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bd7eb-d0d5-4175-a12b-a934902cef20 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.690225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bd7eb-d0d5-4175-a12b-a934902cef20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:52.690533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bd7eb-d0d5-4175-a12b-a934902cef20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:52.692008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bd7eb-d0d5-4175-a12b-a934902cef20 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:52.708914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 267/535] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:52.710240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bd7eb-d0d5-4175-a12b-a934902cef20 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:52.734501 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5676aa-c1d9-4c16-bf73-c1add0752fab None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:52.812692 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bd7eb-d0d5-4175-a12b-a934902cef20 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:52.820801 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:52.825149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7c3f9-ec4a-4c00-b9d3-90d02f604e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:52.826072 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7c3f9-ec4a-4c00-b9d3-90d02f604e93 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.826473 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7c3f9-ec4a-4c00-b9d3-90d02f604e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.826819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7c3f9-ec4a-4c00-b9d3-90d02f604e93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.850787 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7c3f9-ec4a-4c00-b9d3-90d02f604e93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:52.852360 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7c3f9-ec4a-4c00-b9d3-90d02f604e93 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:52.852822 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c3829-4b92-40d6-a1c9-cbc8945c5b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:52.853624 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c3829-4b92-40d6-a1c9-cbc8945c5b35 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.853906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c3829-4b92-40d6-a1c9-cbc8945c5b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.854179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c3829-4b92-40d6-a1c9-cbc8945c5b35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.861419 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c3829-4b92-40d6-a1c9-cbc8945c5b35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:52.862864 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c3829-4b92-40d6-a1c9-cbc8945c5b35 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:52.878465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-db3c3829-4b92-40d6-a1c9-cbc8945c5b35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:52.878465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:52.878465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:52.878465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:52.878465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:52.878465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:52.878465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:52.878465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:52.878465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:52.878465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:52.878465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:52.878465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:52.878465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:52.878465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:52.878465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:52.879329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:52.880160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:52.880160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:52.880160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:52.880160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:52.880160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:52.880160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:52.880160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:52.880160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:52.880160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 268/536] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:52.885494 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c3829-4b92-40d6-a1c9-cbc8945c5b35 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:52.885998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-4bb7c3f9-ec4a-4c00-b9d3-90d02f604e93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:52.885998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:52.885998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:52.885998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:52.885998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:52.885998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:52.885998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:52.885998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:52.885998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:52.885998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:52.885998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:52.885998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:52.885998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:52.885998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:52.885998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:52.886868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:52.887695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:52.887695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:52.887695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:52.887695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:52.887695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:52.887695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:52.887695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:52.887695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:52.889917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 269/537] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:52.895067 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7c3f9-ec4a-4c00-b9d3-90d02f604e93 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:53.057735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c3829-4b92-40d6-a1c9-cbc8945c5b35 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:53.078699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7c3f9-ec4a-4c00-b9d3-90d02f604e93 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:53.110780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:53.125241 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5cb33-dd7d-4bd2-a5dd-ad3a195c4226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:53.126128 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5cb33-dd7d-4bd2-a5dd-ad3a195c4226 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.126554 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5cb33-dd7d-4bd2-a5dd-ad3a195c4226 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.126974 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5cb33-dd7d-4bd2-a5dd-ad3a195c4226 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.127909 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd5cb33-dd7d-4bd2-a5dd-ad3a195c4226 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:53.128385 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd5cb33-dd7d-4bd2-a5dd-ad3a195c4226 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:53.130024 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd5cb33-dd7d-4bd2-a5dd-ad3a195c4226 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:53.148936 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 269/538] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:53.150431 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b4c82-09d1-4d08-a7a6-076114eef22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:53.151257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b4c82-09d1-4d08-a7a6-076114eef22a None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.151705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b4c82-09d1-4d08-a7a6-076114eef22a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.152142 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b4c82-09d1-4d08-a7a6-076114eef22a None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.153109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b4c82-09d1-4d08-a7a6-076114eef22a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:53.153563 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b4c82-09d1-4d08-a7a6-076114eef22a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:53.154505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5cb33-dd7d-4bd2-a5dd-ad3a195c4226 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:53.156447 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b4c82-09d1-4d08-a7a6-076114eef22a None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:53.188816 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:53.192116 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 270/539] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:53.198110 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b4c82-09d1-4d08-a7a6-076114eef22a None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:53.272941 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5cb33-dd7d-4bd2-a5dd-ad3a195c4226 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:53.309095 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39081e3-9d98-4534-b0ba-1a96e1a34efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:53.309999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39081e3-9d98-4534-b0ba-1a96e1a34efb None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.310454 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39081e3-9d98-4534-b0ba-1a96e1a34efb None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.310887 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39081e3-9d98-4534-b0ba-1a96e1a34efb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.311873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39081e3-9d98-4534-b0ba-1a96e1a34efb None admin] RBAC: Adding query filter params (role.id=7fe62c5c771d41c28f790fade1df9357, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=74327a10eaab43e0af02050da1a24f62) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:53.312358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39081e3-9d98-4534-b0ba-1a96e1a34efb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:53.313958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39081e3-9d98-4534-b0ba-1a96e1a34efb None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:53.321063 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 270/540] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:53 2022] GET /identity/v3/role_assignments?user.id=74327a10eaab43e0af02050da1a24f62&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=7fe62c5c771d41c28f790fade1df9357 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:53.427871 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b4c82-09d1-4d08-a7a6-076114eef22a None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:53.465980 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261e842-c57c-46ed-9860-292af3235524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:53.466650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261e842-c57c-46ed-9860-292af3235524 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.466935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261e842-c57c-46ed-9860-292af3235524 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.467209 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261e842-c57c-46ed-9860-292af3235524 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.468063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261e842-c57c-46ed-9860-292af3235524 None admin] RBAC: Adding query filter params (role.id=7fe62c5c771d41c28f790fade1df9357, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=69aef294bb4943c3822ad7d0a3d91fd5) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:53.468403 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261e842-c57c-46ed-9860-292af3235524 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:53.469848 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261e842-c57c-46ed-9860-292af3235524 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:53.477176 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 271/541] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:53 2022] GET /identity/v3/role_assignments?user.id=69aef294bb4943c3822ad7d0a3d91fd5&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=7fe62c5c771d41c28f790fade1df9357 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:53.899357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e45887-8de3-4a27-948b-1896b89bbaf0 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.899876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e45887-8de3-4a27-948b-1896b89bbaf0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.899876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e45887-8de3-4a27-948b-1896b89bbaf0 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.901525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 271/542] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:53.906855 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393eade9-759f-4fa5-8b8f-b56084d561d1 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.907295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393eade9-759f-4fa5-8b8f-b56084d561d1 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.907439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393eade9-759f-4fa5-8b8f-b56084d561d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.917916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa503c3-124d-4ba4-a96f-6af46cdd2863 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.917916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa503c3-124d-4ba4-a96f-6af46cdd2863 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.917916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa503c3-124d-4ba4-a96f-6af46cdd2863 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.918470 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 272/543] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:53.924765 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec830652-9a7a-49b3-a484-93725b372825 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.925052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec830652-9a7a-49b3-a484-93725b372825 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.925330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec830652-9a7a-49b3-a484-93725b372825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.937020 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-393eade9-759f-4fa5-8b8f-b56084d561d1 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:53.938200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-393eade9-759f-4fa5-8b8f-b56084d561d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:53.961374 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ec830652-9a7a-49b3-a484-93725b372825 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:53.962566 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ec830652-9a7a-49b3-a484-93725b372825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:54.013537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:54.057991 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 272/544] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:54.083682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393eade9-759f-4fa5-8b8f-b56084d561d1 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:54.094964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 273/545] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:54.132720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec830652-9a7a-49b3-a484-93725b372825 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:54.178061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:54.195967 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393eade9-759f-4fa5-8b8f-b56084d561d1 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:54.241393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6a311-ef6e-41f0-8522-121cb35314f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:54.242206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6a311-ef6e-41f0-8522-121cb35314f7 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.242488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6a311-ef6e-41f0-8522-121cb35314f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.242766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6a311-ef6e-41f0-8522-121cb35314f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.265441 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6a311-ef6e-41f0-8522-121cb35314f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:54.280680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-64e6a311-ef6e-41f0-8522-121cb35314f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:54.280680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:54.280680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:54.280680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:54.280680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:54.280680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:54.280680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:54.280680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:54.280680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:54.280680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:54.280680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:54.280680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:54.280680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:54.280680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:54.280680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:54.280680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:54.281613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:54.281613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:54.281613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:54.281613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:54.281613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:54.281613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:54.281613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:54.281613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:54.281613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:54.281613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:54.281613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:54.281613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:54.281613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:54.281613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:54.281613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:54.281613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:54.282406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:54.282406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:54.282406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:54.282406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:54.282406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:54.282406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:54.282406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:54.282406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:54.282406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:54.282406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:54.282406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:54.282406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:54.282406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 273/546] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:54.291952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6a311-ef6e-41f0-8522-121cb35314f7 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:54.346985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec830652-9a7a-49b3-a484-93725b372825 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:54.354887 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:54.426917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf7eee-9c87-4e74-9611-140dfa78c3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:54.428312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf7eee-9c87-4e74-9611-140dfa78c3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.428546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf7eee-9c87-4e74-9611-140dfa78c3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.428818 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf7eee-9c87-4e74-9611-140dfa78c3e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.460303 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf7eee-9c87-4e74-9611-140dfa78c3e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:54.469695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-cdcf7eee-9c87-4e74-9611-140dfa78c3e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:54.469695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:54.469695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:54.469695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:54.469695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:54.469695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:54.469695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:54.469695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:54.469695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:54.469695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:54.469695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:54.469695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:54.469695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:54.469695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:54.469695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:54.469695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:54.470592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:54.470592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:54.470592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:54.470592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:54.470592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:54.470592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:54.470592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:54.470592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:54.470592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:54.470592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:54.470592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:54.470592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:54.470592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:54.470592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:54.470592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:54.470592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:54.471360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:54.471360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:54.471360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:54.471360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:54.471360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:54.471360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:54.471360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:54.471360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:54.471360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:54.471360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:54.471360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:54.471360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:54.474082 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 274/547] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:54.475618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb1035-8f00-4d03-a4e7-f6957e730829 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.476125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb1035-8f00-4d03-a4e7-f6957e730829 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.476622 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb1035-8f00-4d03-a4e7-f6957e730829 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.477900 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6a311-ef6e-41f0-8522-121cb35314f7 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:54.482359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 275/548] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:54.483374 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb1035-8f00-4d03-a4e7-f6957e730829 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:54.522907 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf4974-d1a1-4acb-9fcd-aeb23b604c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:54.523585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf4974-d1a1-4acb-9fcd-aeb23b604c19 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.524134 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf4974-d1a1-4acb-9fcd-aeb23b604c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.524446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf4974-d1a1-4acb-9fcd-aeb23b604c19 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.528336 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf4974-d1a1-4acb-9fcd-aeb23b604c19 None admin] RBAC: Adding query filter params (name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:54.528336 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf4974-d1a1-4acb-9fcd-aeb23b604c19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:54.528336 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf4974-d1a1-4acb-9fcd-aeb23b604c19 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:54.536025 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 274/549] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:54.537526 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9f162-290a-49fb-9e91-c0e442a8fbce None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.537882 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9f162-290a-49fb-9e91-c0e442a8fbce None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.538240 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9f162-290a-49fb-9e91-c0e442a8fbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.558973 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-fdb9f162-290a-49fb-9e91-c0e442a8fbce None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:54.561253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-fdb9f162-290a-49fb-9e91-c0e442a8fbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:54.580722 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:54.609836 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb1035-8f00-4d03-a4e7-f6957e730829 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:54.656190 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 275/550] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:54.657390 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9f162-290a-49fb-9e91-c0e442a8fbce None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:54.668819 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8bb50-2b66-45a7-a457-b1eba1cd5bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:54.669873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8bb50-2b66-45a7-a457-b1eba1cd5bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.670330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8bb50-2b66-45a7-a457-b1eba1cd5bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.670799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8bb50-2b66-45a7-a457-b1eba1cd5bb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.671715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8bb50-2b66-45a7-a457-b1eba1cd5bb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:54.672199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8bb50-2b66-45a7-a457-b1eba1cd5bb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:54.674179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8bb50-2b66-45a7-a457-b1eba1cd5bb7 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:54.687691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 276/551] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:54.688881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8bb50-2b66-45a7-a457-b1eba1cd5bb7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:54.696780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:54.777937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9f162-290a-49fb-9e91-c0e442a8fbce None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:54.799913 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8bb50-2b66-45a7-a457-b1eba1cd5bb7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:54.814692 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae05f2a-6cae-4ae7-afd1-4fe84b0cf607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:54.815818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae05f2a-6cae-4ae7-afd1-4fe84b0cf607 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.815929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae05f2a-6cae-4ae7-afd1-4fe84b0cf607 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.816164 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae05f2a-6cae-4ae7-afd1-4fe84b0cf607 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.827755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae05f2a-6cae-4ae7-afd1-4fe84b0cf607 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:54.829344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae05f2a-6cae-4ae7-afd1-4fe84b0cf607 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:54.838311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3185fc-6a20-4ab1-a6e9-38e25b6a111c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:54.839406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3185fc-6a20-4ab1-a6e9-38e25b6a111c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.839885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3185fc-6a20-4ab1-a6e9-38e25b6a111c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.840382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3185fc-6a20-4ab1-a6e9-38e25b6a111c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.848250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:54.849903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-3ae05f2a-6cae-4ae7-afd1-4fe84b0cf607 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:54.849903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:54.849903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:54.849903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:54.849903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:54.849903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:54.849903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:54.849903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:54.849903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:54.849903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:54.849903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:54.849903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:54.849903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:54.849903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:54.849903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:54.851075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:54.852593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:54.852593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:54.852593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:54.852593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:54.852593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 276/552] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:54.852593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae05f2a-6cae-4ae7-afd1-4fe84b0cf607 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:54.855332 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3185fc-6a20-4ab1-a6e9-38e25b6a111c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:54.863843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-ad3185fc-6a20-4ab1-a6e9-38e25b6a111c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:54.863843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:54.863843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:54.863843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:54.863843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:54.863843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:54.863843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:54.863843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:54.863843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:54.863843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:54.863843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:54.863843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:54.863843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:54.863843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:54.863843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:54.863843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:54.864995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:54.864995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:54.864995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:54.864995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:54.864995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:54.864995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:54.864995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:54.864995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:54.864995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:54.864995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:54.864995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:54.864995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:54.864995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:54.864995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:54.864995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:54.864995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:54.866021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:54.866021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:54.866021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:54.866021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:54.866021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:54.866021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:54.866021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:54.866021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:54.866021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:54.866021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:54.866021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:54.866021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:54.868947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 277/553] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:54.870024 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3185fc-6a20-4ab1-a6e9-38e25b6a111c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:54.951182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:54.984713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae05f2a-6cae-4ae7-afd1-4fe84b0cf607 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.006190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3185fc-6a20-4ab1-a6e9-38e25b6a111c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.027695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0b28c-1180-40ef-b92a-6cc8dcaec71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.028530 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0b28c-1180-40ef-b92a-6cc8dcaec71c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.028819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0b28c-1180-40ef-b92a-6cc8dcaec71c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.029097 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0b28c-1180-40ef-b92a-6cc8dcaec71c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.042597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0b28c-1180-40ef-b92a-6cc8dcaec71c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.045326 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0b28c-1180-40ef-b92a-6cc8dcaec71c None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.051048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036eb9b-10ac-4d37-bcfd-00a70edba556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.051723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036eb9b-10ac-4d37-bcfd-00a70edba556 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.052007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036eb9b-10ac-4d37-bcfd-00a70edba556 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.052849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036eb9b-10ac-4d37-bcfd-00a70edba556 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.053631 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036eb9b-10ac-4d37-bcfd-00a70edba556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:55.053946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036eb9b-10ac-4d37-bcfd-00a70edba556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.055485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036eb9b-10ac-4d37-bcfd-00a70edba556 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.067455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-4fc0b28c-1180-40ef-b92a-6cc8dcaec71c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:55.067455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:55.067455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:55.067455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:55.067455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:55.067455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:55.067455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:55.067455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:55.067455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:55.067455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:55.067455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:55.067455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:55.067455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:55.067455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:55.067455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:55.068428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:55.069361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:55.069361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:55.069361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:55.069361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:55.069570 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 278/554] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:55.069570 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036eb9b-10ac-4d37-bcfd-00a70edba556 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.072111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 277/555] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:55.077138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0b28c-1180-40ef-b92a-6cc8dcaec71c None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.150840 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:55.202594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036eb9b-10ac-4d37-bcfd-00a70edba556 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.210053 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0b28c-1180-40ef-b92a-6cc8dcaec71c None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.236876 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:55.244380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9a3dd-a1aa-4b7c-a372-0e3f16ef47c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.245533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9a3dd-a1aa-4b7c-a372-0e3f16ef47c2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.246099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9a3dd-a1aa-4b7c-a372-0e3f16ef47c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.246580 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9a3dd-a1aa-4b7c-a372-0e3f16ef47c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.246874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54a2f5-c960-4ea7-8834-c6d6baf14e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.247674 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54a2f5-c960-4ea7-8834-c6d6baf14e5b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.247960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54a2f5-c960-4ea7-8834-c6d6baf14e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.248237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54a2f5-c960-4ea7-8834-c6d6baf14e5b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.248459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9a3dd-a1aa-4b7c-a372-0e3f16ef47c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:55.248979 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9a3dd-a1aa-4b7c-a372-0e3f16ef47c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.250800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9a3dd-a1aa-4b7c-a372-0e3f16ef47c2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.255898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54a2f5-c960-4ea7-8834-c6d6baf14e5b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.257364 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54a2f5-c960-4ea7-8834-c6d6baf14e5b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.261229 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 279/556] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:55.264421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9a3dd-a1aa-4b7c-a372-0e3f16ef47c2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.282822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-8c54a2f5-c960-4ea7-8834-c6d6baf14e5b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:10:55.282822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:55.282822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:55.282822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:55.282822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:55.282822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:55.282822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:55.282822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:55.282822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:55.282822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:55.282822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:55.282822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:55.282822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:55.282822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:55.282822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:55.282822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:55.283790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:55.285266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:55.285266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:55.285266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:55.285266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:55.285266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:55.285266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:55.285266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:55.285266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:55.285266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:10:55.285266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:55.289535 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 278/557] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:55.290469 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54a2f5-c960-4ea7-8834-c6d6baf14e5b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.376335 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9a3dd-a1aa-4b7c-a372-0e3f16ef47c2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.400321 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:55.413141 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54a2f5-c960-4ea7-8834-c6d6baf14e5b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.413330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbf4b9-dab0-4359-8756-9c621989b1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.413989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbf4b9-dab0-4359-8756-9c621989b1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.414279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbf4b9-dab0-4359-8756-9c621989b1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.414546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbf4b9-dab0-4359-8756-9c621989b1b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.415273 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbbf4b9-dab0-4359-8756-9c621989b1b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:55.415704 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbbf4b9-dab0-4359-8756-9c621989b1b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.417282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbbf4b9-dab0-4359-8756-9c621989b1b1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.431348 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 280/558] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:55.432255 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbf4b9-dab0-4359-8756-9c621989b1b1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.454049 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82aef0-416f-4a37-8341-8b01841a3ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.455187 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82aef0-416f-4a37-8341-8b01841a3ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.455674 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82aef0-416f-4a37-8341-8b01841a3ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.456193 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82aef0-416f-4a37-8341-8b01841a3ac1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.467820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82aef0-416f-4a37-8341-8b01841a3ac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.469768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82aef0-416f-4a37-8341-8b01841a3ac1 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.490425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-bf82aef0-416f-4a37-8341-8b01841a3ac1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:55.490425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:55.490425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:55.490425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:55.490425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:55.490425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:55.490425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:55.490425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:55.490425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:55.490425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:55.490425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:55.490425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:55.490425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:55.490425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:55.490425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:55.491445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:55.492341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:55.492341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:55.492341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:55.492341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:55.499018 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 279/559] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:55.500244 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82aef0-416f-4a37-8341-8b01841a3ac1 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.544604 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:55.547722 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbf4b9-dab0-4359-8756-9c621989b1b1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.583011 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908609e-479f-4ec5-a0c4-78439e5e6c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.584267 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908609e-479f-4ec5-a0c4-78439e5e6c12 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.584577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908609e-479f-4ec5-a0c4-78439e5e6c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.584856 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908609e-479f-4ec5-a0c4-78439e5e6c12 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.585703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908609e-479f-4ec5-a0c4-78439e5e6c12 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:55.586013 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908609e-479f-4ec5-a0c4-78439e5e6c12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.587463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908609e-479f-4ec5-a0c4-78439e5e6c12 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.604895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 281/560] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:55.610042 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908609e-479f-4ec5-a0c4-78439e5e6c12 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.614778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82aef0-416f-4a37-8341-8b01841a3ac1 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.651062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5c035-9287-48de-b26c-4644b0601d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.652466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5c035-9287-48de-b26c-4644b0601d4e None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.652752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5c035-9287-48de-b26c-4644b0601d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.653026 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5c035-9287-48de-b26c-4644b0601d4e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.653927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:55.665056 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5c035-9287-48de-b26c-4644b0601d4e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.667162 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5c035-9287-48de-b26c-4644b0601d4e None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.687237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-a6b5c035-9287-48de-b26c-4644b0601d4e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:10:55.687237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:55.687237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:55.687237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:55.687237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:55.687237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:55.687237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:55.687237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:55.687237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:55.687237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:55.687237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:55.687237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:55.687237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:55.687237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:55.687237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:55.687237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:55.688321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:55.689270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:55.689270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:55.689270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:55.689270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:55.689270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:55.689270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:55.689270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:55.689270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:55.689270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:10:55.689270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:55.689270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 280/561] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:55.689850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e7fdb-a820-4df5-92cd-08415fdab4f3 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.690797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e7fdb-a820-4df5-92cd-08415fdab4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.690797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e7fdb-a820-4df5-92cd-08415fdab4f3 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.691807 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 281/562] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:55.696333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e7fdb-a820-4df5-92cd-08415fdab4f3 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.721861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908609e-479f-4ec5-a0c4-78439e5e6c12 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.761443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb0222-2573-4215-b5ba-164ce13be525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.762107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb0222-2573-4215-b5ba-164ce13be525 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.762389 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb0222-2573-4215-b5ba-164ce13be525 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.762817 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb0222-2573-4215-b5ba-164ce13be525 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.764241 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb0222-2573-4215-b5ba-164ce13be525 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:55.764550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb0222-2573-4215-b5ba-164ce13be525 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.765919 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb0222-2573-4215-b5ba-164ce13be525 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.780159 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 282/563] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:55.780947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb0222-2573-4215-b5ba-164ce13be525 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.799075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:55.804536 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e7fdb-a820-4df5-92cd-08415fdab4f3 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.838467 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11707de3-c355-4df7-b0e7-d74437624af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.839256 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11707de3-c355-4df7-b0e7-d74437624af8 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.839535 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11707de3-c355-4df7-b0e7-d74437624af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.839806 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11707de3-c355-4df7-b0e7-d74437624af8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.849513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11707de3-c355-4df7-b0e7-d74437624af8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.850958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11707de3-c355-4df7-b0e7-d74437624af8 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.869451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-11707de3-c355-4df7-b0e7-d74437624af8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:55.869451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:55.869451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:55.869451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:55.869451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:55.869451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:55.869451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:55.869451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:55.869451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:55.869451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:55.869451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:55.869451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:55.869451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:55.869451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:55.869451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:55.875028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:55.876138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:55.876138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:55.876138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:55.876138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:55.876138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 282/564] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:55.876138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ac985-44cc-4b5d-ada5-0351585e49d6 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.876138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ac985-44cc-4b5d-ada5-0351585e49d6 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.876138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ac985-44cc-4b5d-ada5-0351585e49d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.883460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:55.889842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-af4ac985-44cc-4b5d-ada5-0351585e49d6 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:55.891031 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-af4ac985-44cc-4b5d-ada5-0351585e49d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:55.891298 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb0222-2573-4215-b5ba-164ce13be525 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.935584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30a4fc-2732-4dac-acab-19b8dbb361c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.936246 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30a4fc-2732-4dac-acab-19b8dbb361c8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.936554 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30a4fc-2732-4dac-acab-19b8dbb361c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.936830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30a4fc-2732-4dac-acab-19b8dbb361c8 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.937670 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30a4fc-2732-4dac-acab-19b8dbb361c8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:55.937979 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30a4fc-2732-4dac-acab-19b8dbb361c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.939431 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30a4fc-2732-4dac-acab-19b8dbb361c8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.957280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 283/565] 10.222.0.43 () {56 vars in 1135 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:55.958102 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30a4fc-2732-4dac-acab-19b8dbb361c8 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.971247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 283/566] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:55.971247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26d008-1e8e-4f45-957d-272115e6bfee None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.971247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26d008-1e8e-4f45-957d-272115e6bfee None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.971247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26d008-1e8e-4f45-957d-272115e6bfee None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.971247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 284/567] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:55.971247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26d008-1e8e-4f45-957d-272115e6bfee None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.053278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:56.060024 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30a4fc-2732-4dac-acab-19b8dbb361c8 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.073582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26d008-1e8e-4f45-957d-272115e6bfee None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.094619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d635222-7927-4e31-9c71-10a537f245c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.095411 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d635222-7927-4e31-9c71-10a537f245c9 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.095690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d635222-7927-4e31-9c71-10a537f245c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.095960 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d635222-7927-4e31-9c71-10a537f245c9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.103601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d635222-7927-4e31-9c71-10a537f245c9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:56.105084 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d635222-7927-4e31-9c71-10a537f245c9 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:56.108565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4a216-6f37-4535-896a-657b662713b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.109222 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4a216-6f37-4535-896a-657b662713b7 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.109506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4a216-6f37-4535-896a-657b662713b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.109784 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4a216-6f37-4535-896a-657b662713b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.110502 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4a216-6f37-4535-896a-657b662713b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:56.110810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4a216-6f37-4535-896a-657b662713b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:56.112150 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4a216-6f37-4535-896a-657b662713b7 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:56.119895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-4d635222-7927-4e31-9c71-10a537f245c9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:10:56.119895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:56.119895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:56.119895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:56.119895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:56.119895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:56.119895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:56.119895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:56.119895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:56.119895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:56.119895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:56.119895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:56.119895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:56.119895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:56.119895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:56.119895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:56.121023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:56.126929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:56.126929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:56.126929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:56.126929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:56.126929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:56.126929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:56.126929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:56.126929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:56.126929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:10:56.126929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:56.126929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 284/568] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:56.126929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d635222-7927-4e31-9c71-10a537f245c9 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.136602 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 285/569] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:56.136602 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea06e03-9b93-4c88-a257-22945c14b0c2 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.136602 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea06e03-9b93-4c88-a257-22945c14b0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.136602 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea06e03-9b93-4c88-a257-22945c14b0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.140135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:56.151661 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-6ea06e03-9b93-4c88-a257-22945c14b0c2 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:56.152886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6ea06e03-9b93-4c88-a257-22945c14b0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:56.218623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 286/570] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:56.219449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea06e03-9b93-4c88-a257-22945c14b0c2 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.220147 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d635222-7927-4e31-9c71-10a537f245c9 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.258951 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69cc7e-351d-467a-b25e-ad2f14fda7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.259596 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69cc7e-351d-467a-b25e-ad2f14fda7ce None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.259878 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69cc7e-351d-467a-b25e-ad2f14fda7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.260155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69cc7e-351d-467a-b25e-ad2f14fda7ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.286269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:56.318957 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea06e03-9b93-4c88-a257-22945c14b0c2 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.331632 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69cc7e-351d-467a-b25e-ad2f14fda7ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:56.333136 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69cc7e-351d-467a-b25e-ad2f14fda7ce None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:56.351376 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:56.354421 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a59d5a-a223-4cf1-ae73-a7cebac59fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.355320 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a59d5a-a223-4cf1-ae73-a7cebac59fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.355604 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a59d5a-a223-4cf1-ae73-a7cebac59fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.355880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a59d5a-a223-4cf1-ae73-a7cebac59fb7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.366012 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a59d5a-a223-4cf1-ae73-a7cebac59fb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:56.373721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-a4a59d5a-a223-4cf1-ae73-a7cebac59fb7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:56.373721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:56.373721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:56.373721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:56.373721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:56.373721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:56.373721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:56.373721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:56.373721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:56.373721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:56.373721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:56.373721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:56.373721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:56.373721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:56.373721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:56.373721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:56.374653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:56.374653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:56.374653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:56.374653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:56.374653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:56.374653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:56.374653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:56.374653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:56.374653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:56.374653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:56.374653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:56.374653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:56.374653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:56.374653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:56.374653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:56.374653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:56.375463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:56.375463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:56.375463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:56.375463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:56.375463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:56.375463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:56.375463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:56.375463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:56.375463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:56.375463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:56.375463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:56.375463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:56.375463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 287/571] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:56.376223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a59d5a-a223-4cf1-ae73-a7cebac59fb7 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.434847 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 285/572] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:56.435614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69cc7e-351d-467a-b25e-ad2f14fda7ce None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.482123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a59d5a-a223-4cf1-ae73-a7cebac59fb7 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.513593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:56.521498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e722aed-efbe-4ab7-a541-0e1d2c7c8eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.522182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e722aed-efbe-4ab7-a541-0e1d2c7c8eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.522467 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e722aed-efbe-4ab7-a541-0e1d2c7c8eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.522747 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e722aed-efbe-4ab7-a541-0e1d2c7c8eb1 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.523603 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e722aed-efbe-4ab7-a541-0e1d2c7c8eb1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:56.523916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e722aed-efbe-4ab7-a541-0e1d2c7c8eb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:56.525372 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e722aed-efbe-4ab7-a541-0e1d2c7c8eb1 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:56.534349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69cc7e-351d-467a-b25e-ad2f14fda7ce None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.542470 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 288/573] 10.222.0.43 () {56 vars in 1137 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:56.543264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e722aed-efbe-4ab7-a541-0e1d2c7c8eb1 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.556308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:56.574545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0bb19-15dd-492c-9f51-483b90c54299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.575275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0bb19-15dd-492c-9f51-483b90c54299 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.575564 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0bb19-15dd-492c-9f51-483b90c54299 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.575837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0bb19-15dd-492c-9f51-483b90c54299 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.582497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0bb19-15dd-492c-9f51-483b90c54299 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:56.584466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0bb19-15dd-492c-9f51-483b90c54299 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:56.598015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-67f0bb19-15dd-492c-9f51-483b90c54299 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:56.598015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:56.598015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:56.598015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:56.598015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:56.598015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:56.598015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:56.598015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:56.598015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:56.598015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:56.598015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:56.598015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:56.598015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:56.598015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:56.598015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:56.599604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:56.600721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:56.600721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:56.600721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:56.600721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:56.600721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:56.600721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:56.600721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:56.600721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:56.600721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 286/574] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:56.600721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0bb19-15dd-492c-9f51-483b90c54299 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.647492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e722aed-efbe-4ab7-a541-0e1d2c7c8eb1 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.682494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc6806-55c2-4d65-ab34-2950d6960bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.683572 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc6806-55c2-4d65-ab34-2950d6960bbe None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.683813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc6806-55c2-4d65-ab34-2950d6960bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.684080 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc6806-55c2-4d65-ab34-2950d6960bbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.694036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc6806-55c2-4d65-ab34-2950d6960bbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:56.701704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-48bc6806-55c2-4d65-ab34-2950d6960bbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:56.701704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:56.701704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:56.701704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:56.701704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:56.701704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:56.701704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:56.701704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:56.701704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:56.701704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:56.701704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:56.701704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:56.701704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:56.701704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:56.701704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:56.701704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:56.702811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:56.702811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:56.702811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:56.702811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:56.702811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:56.702811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:56.702811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:56.702811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:56.702811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:56.702811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:56.702811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:56.702811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:56.702811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:56.702811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:56.702811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:56.702811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:56.703657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:56.703657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:56.703657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:56.703657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:56.703657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:56.703657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:56.703657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:56.703657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:56.703657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:56.703657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:56.703657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:56.703657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:56.703657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 289/575] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:56.704369 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0bb19-15dd-492c-9f51-483b90c54299 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.704444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc6806-55c2-4d65-ab34-2950d6960bbe None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.744894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b4245-fcd3-43d4-ae26-92268def9909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.745560 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b4245-fcd3-43d4-ae26-92268def9909 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.745845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b4245-fcd3-43d4-ae26-92268def9909 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.746122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b4245-fcd3-43d4-ae26-92268def9909 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.746878 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b4245-fcd3-43d4-ae26-92268def9909 None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:56.747188 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b4245-fcd3-43d4-ae26-92268def9909 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:56.748606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b4245-fcd3-43d4-ae26-92268def9909 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:56.754055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:56.756677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 287/576] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:56.757472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b4245-fcd3-43d4-ae26-92268def9909 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.801086 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:56.807271 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc6806-55c2-4d65-ab34-2950d6960bbe None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.841425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8dfd8-d5bd-46c8-b279-abcd821647eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.842268 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8dfd8-d5bd-46c8-b279-abcd821647eb None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.842519 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8dfd8-d5bd-46c8-b279-abcd821647eb None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.842794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8dfd8-d5bd-46c8-b279-abcd821647eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.854260 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8dfd8-d5bd-46c8-b279-abcd821647eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:56.855867 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8dfd8-d5bd-46c8-b279-abcd821647eb None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:56.862752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b4245-fcd3-43d4-ae26-92268def9909 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.873124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-4fd8dfd8-d5bd-46c8-b279-abcd821647eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:56.873124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:56.873124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:56.873124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:56.873124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:56.873124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:56.873124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:56.873124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:56.873124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:56.873124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:56.873124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:56.873124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:56.873124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:56.873124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:56.873124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:56.874143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:56.879791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:56.879791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:56.879791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:56.879791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:56.879791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 290/577] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:56.879791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8dfd8-d5bd-46c8-b279-abcd821647eb None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.897714 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f701ad-f4fd-4470-b9ae-0851ed85aa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.898498 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f701ad-f4fd-4470-b9ae-0851ed85aa1e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.898782 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f701ad-f4fd-4470-b9ae-0851ed85aa1e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.899058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f701ad-f4fd-4470-b9ae-0851ed85aa1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.910283 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f701ad-f4fd-4470-b9ae-0851ed85aa1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:56.912252 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f701ad-f4fd-4470-b9ae-0851ed85aa1e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:56.929892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-54f701ad-f4fd-4470-b9ae-0851ed85aa1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:56.929892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:56.929892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:56.929892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:56.929892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:56.929892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:56.929892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:56.929892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:56.929892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:56.929892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:56.929892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:56.929892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:56.929892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:56.929892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:56.929892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:56.930837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:56.931753 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:56.931753 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:56.931753 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:56.931753 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:56.931753 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 288/578] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:56.932461 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f701ad-f4fd-4470-b9ae-0851ed85aa1e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.977871 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:56.980755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8dfd8-d5bd-46c8-b279-abcd821647eb None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:57.015481 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526a46-bd10-4e90-bb6a-bdb879dcbc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:57.016143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526a46-bd10-4e90-bb6a-bdb879dcbc36 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:57.016462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526a46-bd10-4e90-bb6a-bdb879dcbc36 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:57.016753 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526a46-bd10-4e90-bb6a-bdb879dcbc36 None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:57.017547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8526a46-bd10-4e90-bb6a-bdb879dcbc36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:57.017864 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8526a46-bd10-4e90-bb6a-bdb879dcbc36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:57.019300 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8526a46-bd10-4e90-bb6a-bdb879dcbc36 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:57.023503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:57.032158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 291/579] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:57.032991 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526a46-bd10-4e90-bb6a-bdb879dcbc36 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:57.033242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f701ad-f4fd-4470-b9ae-0851ed85aa1e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:57.068162 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a288726-0576-4f55-a8ba-033ba445d5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:57.068854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a288726-0576-4f55-a8ba-033ba445d5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:57.069164 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a288726-0576-4f55-a8ba-033ba445d5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:57.069440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a288726-0576-4f55-a8ba-033ba445d5c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:57.070189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a288726-0576-4f55-a8ba-033ba445d5c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:57.070499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a288726-0576-4f55-a8ba-033ba445d5c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:57.071869 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a288726-0576-4f55-a8ba-033ba445d5c2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:57.080115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 289/580] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:57.080929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a288726-0576-4f55-a8ba-033ba445d5c2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:57.134097 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526a46-bd10-4e90-bb6a-bdb879dcbc36 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:57.167980 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55957cc3-e8d9-4054-925f-acf7237c2e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:57.168809 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55957cc3-e8d9-4054-925f-acf7237c2e20 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:57.169093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55957cc3-e8d9-4054-925f-acf7237c2e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:57.169367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55957cc3-e8d9-4054-925f-acf7237c2e20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:57.179619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a288726-0576-4f55-a8ba-033ba445d5c2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:57.180057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55957cc3-e8d9-4054-925f-acf7237c2e20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:57.181640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55957cc3-e8d9-4054-925f-acf7237c2e20 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:57.200595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-55957cc3-e8d9-4054-925f-acf7237c2e20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:57.200595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:57.200595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:57.200595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:57.200595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:57.200595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:57.200595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:57.200595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:57.200595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:57.200595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:57.200595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:57.200595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:57.200595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:57.200595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:57.200595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:57.201565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:57.202452 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:57.202452 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:57.202452 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:57.202452 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:57.202452 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 292/581] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:57.202854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55957cc3-e8d9-4054-925f-acf7237c2e20 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:57.213943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc76b18-052b-4a0b-b7b7-85e0c4964b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:57.214596 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc76b18-052b-4a0b-b7b7-85e0c4964b2a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:57.214875 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc76b18-052b-4a0b-b7b7-85e0c4964b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:57.215147 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc76b18-052b-4a0b-b7b7-85e0c4964b2a None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:57.215863 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc76b18-052b-4a0b-b7b7-85e0c4964b2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:57.216169 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc76b18-052b-4a0b-b7b7-85e0c4964b2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:57.217132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:57.217740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc76b18-052b-4a0b-b7b7-85e0c4964b2a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:57.229911 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 290/582] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:57.230676 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc76b18-052b-4a0b-b7b7-85e0c4964b2a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:57.246532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:57.307561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55957cc3-e8d9-4054-925f-acf7237c2e20 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:57.332949 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc76b18-052b-4a0b-b7b7-85e0c4964b2a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:57.342960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b78afb-3a45-478b-b442-545c4c816dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:57.343621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b78afb-3a45-478b-b442-545c4c816dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:57.343894 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b78afb-3a45-478b-b442-545c4c816dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:57.344173 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b78afb-3a45-478b-b442-545c4c816dd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:57.344941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b78afb-3a45-478b-b442-545c4c816dd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:57.345250 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b78afb-3a45-478b-b442-545c4c816dd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:57.346597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b78afb-3a45-478b-b442-545c4c816dd1 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:57.358709 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 293/583] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:57.359536 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b78afb-3a45-478b-b442-545c4c816dd1 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:57.370554 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eaae6f-8948-48f0-94a5-0c556dc7daca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:57.371447 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaae6f-8948-48f0-94a5-0c556dc7daca None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:57.371734 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaae6f-8948-48f0-94a5-0c556dc7daca None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:57.372015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaae6f-8948-48f0-94a5-0c556dc7daca None admin] PATH_INFO: `/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/267b178236e046e4bc694b2da6f86eb0/roles/7fe62c5c771d41c28f790fade1df9357` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:57.386659 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eaae6f-8948-48f0-94a5-0c556dc7daca None admin] RBAC: Authorizing `identity:create_grant(project_id=5e2a50ebbb48417c911df499d98a0157, user_id=267b178236e046e4bc694b2da6f86eb0, role_id=7fe62c5c771d41c28f790fade1df9357)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:57.388047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eaae6f-8948-48f0-94a5-0c556dc7daca None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:57.403058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 291/584] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:57 2022] PUT /identity/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/267b178236e046e4bc694b2da6f86eb0/roles/7fe62c5c771d41c28f790fade1df9357 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:57.403634 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eaae6f-8948-48f0-94a5-0c556dc7daca None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:57.438131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:57.474891 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b78afb-3a45-478b-b442-545c4c816dd1 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:57.491240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:57.515244 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79564757-60ce-4eed-ac64-0b7f5a6df3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:57.516067 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79564757-60ce-4eed-ac64-0b7f5a6df3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:57.516383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79564757-60ce-4eed-ac64-0b7f5a6df3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:57.516667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79564757-60ce-4eed-ac64-0b7f5a6df3d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:57.530747 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79564757-60ce-4eed-ac64-0b7f5a6df3d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:57.532225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79564757-60ce-4eed-ac64-0b7f5a6df3d4 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:57.538831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eaae6f-8948-48f0-94a5-0c556dc7daca None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:57.553751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-79564757-60ce-4eed-ac64-0b7f5a6df3d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:57.553751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:57.553751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:57.553751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:57.553751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:57.553751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:57.553751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:57.553751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:57.553751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:57.553751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:57.553751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:57.553751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:57.553751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:57.553751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:57.553751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:57.554702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:57.555539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:57.555539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:57.555539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:57.555539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:57.555539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 294/585] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:57.555811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79564757-60ce-4eed-ac64-0b7f5a6df3d4 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:57.580744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03d0b9-1acc-4b12-8905-0c0750fd4773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:57.581431 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03d0b9-1acc-4b12-8905-0c0750fd4773 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:57.581772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03d0b9-1acc-4b12-8905-0c0750fd4773 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:57.582053 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03d0b9-1acc-4b12-8905-0c0750fd4773 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:57.582797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03d0b9-1acc-4b12-8905-0c0750fd4773 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:57.583108 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03d0b9-1acc-4b12-8905-0c0750fd4773 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:57.585063 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03d0b9-1acc-4b12-8905-0c0750fd4773 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:57.596834 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 292/586] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:57.597731 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03d0b9-1acc-4b12-8905-0c0750fd4773 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:57.679241 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79564757-60ce-4eed-ac64-0b7f5a6df3d4 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:57.708178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:57.721352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03d0b9-1acc-4b12-8905-0c0750fd4773 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:57.721807 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20689e05-400a-4437-8e49-46ac9def1f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:57.722687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20689e05-400a-4437-8e49-46ac9def1f67 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:57.723066 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20689e05-400a-4437-8e49-46ac9def1f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:57.723412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20689e05-400a-4437-8e49-46ac9def1f67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:57.731487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20689e05-400a-4437-8e49-46ac9def1f67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:57.734190 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20689e05-400a-4437-8e49-46ac9def1f67 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:57.749417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:57.756145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-20689e05-400a-4437-8e49-46ac9def1f67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:57.756145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:57.756145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:57.756145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:57.756145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:57.756145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:57.756145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:57.756145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:57.756145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:57.756145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:57.756145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:57.756145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:57.756145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:57.756145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:57.756145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:57.757337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:57.763561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:57.763561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:57.763561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:57.763561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:57.763561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:57.763561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:57.763561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:57.763561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:57.763561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 295/587] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:57.763561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20689e05-400a-4437-8e49-46ac9def1f67 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:57.770455 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660c92d-88d1-46a3-998b-ac114cf564c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:57.772235 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660c92d-88d1-46a3-998b-ac114cf564c4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:57.772390 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660c92d-88d1-46a3-998b-ac114cf564c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:57.772589 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660c92d-88d1-46a3-998b-ac114cf564c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:57.780395 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660c92d-88d1-46a3-998b-ac114cf564c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:57.781975 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660c92d-88d1-46a3-998b-ac114cf564c4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:57.797325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-c660c92d-88d1-46a3-998b-ac114cf564c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:57.797325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:57.797325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:57.797325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:57.797325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:57.797325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:57.797325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:57.797325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:57.797325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:57.797325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:57.797325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:57.797325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:57.797325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:57.797325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:57.797325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:57.798402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:57.799501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:57.799501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:57.799501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:57.799501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:57.799501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:57.799501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:57.799501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:57.799501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:57.800605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 293/588] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:57.801630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660c92d-88d1-46a3-998b-ac114cf564c4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:57.866549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20689e05-400a-4437-8e49-46ac9def1f67 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:57.905186 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a991d-e604-4434-8e90-3c17a52a1954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:57.905863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a991d-e604-4434-8e90-3c17a52a1954 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:57.906157 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a991d-e604-4434-8e90-3c17a52a1954 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:57.906460 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a991d-e604-4434-8e90-3c17a52a1954 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:57.907200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a991d-e604-4434-8e90-3c17a52a1954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:57.907512 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a991d-e604-4434-8e90-3c17a52a1954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:57.908899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a991d-e604-4434-8e90-3c17a52a1954 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:57.910277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660c92d-88d1-46a3-998b-ac114cf564c4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:57.926406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 296/589] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:57.926406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a991d-e604-4434-8e90-3c17a52a1954 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:57.953272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b40d20-07e6-4c1f-aae9-d10d131e995b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:57.954143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b40d20-07e6-4c1f-aae9-d10d131e995b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:57.954535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b40d20-07e6-4c1f-aae9-d10d131e995b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:57.954913 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b40d20-07e6-4c1f-aae9-d10d131e995b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:57.962922 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b40d20-07e6-4c1f-aae9-d10d131e995b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:57.964531 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b40d20-07e6-4c1f-aae9-d10d131e995b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:57.975078 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:57.979770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-e6b40d20-07e6-4c1f-aae9-d10d131e995b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:10:57.979770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:57.979770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:57.979770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:57.979770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:57.979770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:57.979770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:57.979770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:57.979770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:57.979770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:57.979770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:57.979770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:57.979770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:57.979770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:57.979770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:57.979770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:57.980923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:57.982060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:57.982060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:57.982060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:57.982060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:57.982060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:57.982060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:57.982060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:57.982060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:57.982060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:10:57.982060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:57.983202 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 294/590] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:57.983976 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b40d20-07e6-4c1f-aae9-d10d131e995b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:58.036461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a991d-e604-4434-8e90-3c17a52a1954 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:58.037912 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:58.071994 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e635c0c-f287-4575-818e-c293b971e845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:58.072743 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e635c0c-f287-4575-818e-c293b971e845 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:58.073030 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e635c0c-f287-4575-818e-c293b971e845 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:58.073307 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e635c0c-f287-4575-818e-c293b971e845 None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:58.074095 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e635c0c-f287-4575-818e-c293b971e845 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:58.074404 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e635c0c-f287-4575-818e-c293b971e845 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:58.075824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e635c0c-f287-4575-818e-c293b971e845 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:58.094050 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 297/591] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:58.094050 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e635c0c-f287-4575-818e-c293b971e845 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:58.104449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b40d20-07e6-4c1f-aae9-d10d131e995b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:58.141274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392a9dc-2670-428d-be47-030700cfdce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:58.141995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392a9dc-2670-428d-be47-030700cfdce1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:58.142275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392a9dc-2670-428d-be47-030700cfdce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:58.142546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392a9dc-2670-428d-be47-030700cfdce1 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:58.143388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392a9dc-2670-428d-be47-030700cfdce1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:58.143703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392a9dc-2670-428d-be47-030700cfdce1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:58.145161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392a9dc-2670-428d-be47-030700cfdce1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:58.157800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 295/592] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:58.157800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392a9dc-2670-428d-be47-030700cfdce1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:58.205898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e635c0c-f287-4575-818e-c293b971e845 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:58.209400 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:58.246132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e666f3-18c5-4df8-94c2-6998b417e041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:58.246934 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e666f3-18c5-4df8-94c2-6998b417e041 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:58.247221 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e666f3-18c5-4df8-94c2-6998b417e041 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:58.247498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e666f3-18c5-4df8-94c2-6998b417e041 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:58.255913 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e666f3-18c5-4df8-94c2-6998b417e041 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:58.257852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e666f3-18c5-4df8-94c2-6998b417e041 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:58.269369 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392a9dc-2670-428d-be47-030700cfdce1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:58.272960 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:58.273461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-e8e666f3-18c5-4df8-94c2-6998b417e041 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:10:58.273461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:58.273461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:58.273461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:58.273461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:58.273461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:58.273461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:58.273461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:58.273461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:58.273461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:58.273461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:58.273461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:58.273461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:58.273461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:58.273461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:58.273461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:58.274362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:58.275306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:58.275306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:58.275306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:58.275306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:58.275306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:58.275306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:58.275306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:58.275306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:58.275306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:10:58.275306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:58.275306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 298/593] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:58.275940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e666f3-18c5-4df8-94c2-6998b417e041 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:58.316806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94636f4c-4e4a-499a-b15e-2cadf7603fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:58.317489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94636f4c-4e4a-499a-b15e-2cadf7603fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:58.317830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94636f4c-4e4a-499a-b15e-2cadf7603fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:58.318113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94636f4c-4e4a-499a-b15e-2cadf7603fa5 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:58.318970 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94636f4c-4e4a-499a-b15e-2cadf7603fa5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:58.319286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94636f4c-4e4a-499a-b15e-2cadf7603fa5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:58.320747 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94636f4c-4e4a-499a-b15e-2cadf7603fa5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:58.346432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 296/594] 10.222.0.43 () {56 vars in 1139 bytes} [Wed Jul 20 19:10:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:58.346432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94636f4c-4e4a-499a-b15e-2cadf7603fa5 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:58.383697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e666f3-18c5-4df8-94c2-6998b417e041 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:58.420128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88a395-29ec-43f9-958a-abd75ce6c842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:58.420883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88a395-29ec-43f9-958a-abd75ce6c842 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:58.421189 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88a395-29ec-43f9-958a-abd75ce6c842 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:58.421460 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88a395-29ec-43f9-958a-abd75ce6c842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:58.422270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88a395-29ec-43f9-958a-abd75ce6c842 None admin] RBAC: Adding query filter params (role.id=7fe62c5c771d41c28f790fade1df9357, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=1c944eab8a8948749bb8375787216b09) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:58.422582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88a395-29ec-43f9-958a-abd75ce6c842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:58.424002 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88a395-29ec-43f9-958a-abd75ce6c842 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:58.440337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 299/595] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:58 2022] GET /identity/v3/role_assignments?user.id=1c944eab8a8948749bb8375787216b09&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=7fe62c5c771d41c28f790fade1df9357 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:58.440337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88a395-29ec-43f9-958a-abd75ce6c842 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:58.472212 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94636f4c-4e4a-499a-b15e-2cadf7603fa5 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:58.477162 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:58.523584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d004b46-8837-4b1c-9a24-86c65704276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:58.524641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d004b46-8837-4b1c-9a24-86c65704276e None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:58.525057 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d004b46-8837-4b1c-9a24-86c65704276e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:58.525427 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d004b46-8837-4b1c-9a24-86c65704276e None admin] PATH_INFO: `/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/d3e28557896e41c893f0de2590bc96d7/roles/7fe62c5c771d41c28f790fade1df9357` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:58.545858 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d004b46-8837-4b1c-9a24-86c65704276e None admin] RBAC: Authorizing `identity:create_grant(project_id=5e2a50ebbb48417c911df499d98a0157, user_id=d3e28557896e41c893f0de2590bc96d7, role_id=7fe62c5c771d41c28f790fade1df9357)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:58.547539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d004b46-8837-4b1c-9a24-86c65704276e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:58.573654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 297/596] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:58 2022] PUT /identity/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/d3e28557896e41c893f0de2590bc96d7/roles/7fe62c5c771d41c28f790fade1df9357 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:58.575379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d004b46-8837-4b1c-9a24-86c65704276e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:58.580732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88a395-29ec-43f9-958a-abd75ce6c842 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:58.596599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:58.627066 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22a983-035d-4db8-b8c4-d950c833c909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:58.628249 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22a983-035d-4db8-b8c4-d950c833c909 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:58.628775 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22a983-035d-4db8-b8c4-d950c833c909 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:58.629254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22a983-035d-4db8-b8c4-d950c833c909 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:58.630387 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22a983-035d-4db8-b8c4-d950c833c909 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:58.630940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22a983-035d-4db8-b8c4-d950c833c909 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:58.632664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22a983-035d-4db8-b8c4-d950c833c909 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:58.660852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 300/597] 10.222.0.43 () {56 vars in 1137 bytes} [Wed Jul 20 19:10:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:58.667370 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22a983-035d-4db8-b8c4-d950c833c909 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:58.719307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d004b46-8837-4b1c-9a24-86c65704276e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:58.776029 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf6f20-1134-478c-ae8c-f22918fd9606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:58.777205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf6f20-1134-478c-ae8c-f22918fd9606 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:58.777691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf6f20-1134-478c-ae8c-f22918fd9606 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:58.778167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf6f20-1134-478c-ae8c-f22918fd9606 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:58.782117 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:58.795445 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf6f20-1134-478c-ae8c-f22918fd9606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:58.797280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf6f20-1134-478c-ae8c-f22918fd9606 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:58.801041 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22a983-035d-4db8-b8c4-d950c833c909 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:58.817115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-79cf6f20-1134-478c-ae8c-f22918fd9606 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:58.817115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:58.817115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:58.817115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:58.817115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:58.817115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:58.817115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:58.817115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:58.817115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:58.817115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:58.817115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:58.817115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:58.817115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:58.817115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:58.817115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:58.818139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:58.819087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:58.819087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:58.819087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:58.819087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:58.821369 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 298/598] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:58.827565 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf6f20-1134-478c-ae8c-f22918fd9606 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:58.845720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4a305-5aef-4a10-845b-ce43edc6dcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:58.846999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4a305-5aef-4a10-845b-ce43edc6dcd6 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:58.847294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4a305-5aef-4a10-845b-ce43edc6dcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:58.847570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4a305-5aef-4a10-845b-ce43edc6dcd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:58.863711 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4a305-5aef-4a10-845b-ce43edc6dcd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:58.865894 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4a305-5aef-4a10-845b-ce43edc6dcd6 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:58.889520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-f3c4a305-5aef-4a10-845b-ce43edc6dcd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:58.889520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:58.889520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:58.889520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:58.889520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:58.889520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:58.889520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:58.889520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:58.889520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:58.889520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:58.889520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:58.889520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:58.889520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:58.889520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:58.889520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:58.890466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:58.891358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:58.891358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:58.891358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:58.891358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:58.891358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 301/599] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:58.898008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4a305-5aef-4a10-845b-ce43edc6dcd6 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:58.908742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:58.959646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf6f20-1134-478c-ae8c-f22918fd9606 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:59.010913 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed6730-a27e-4f0c-b16d-b7758dad7192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:59.012919 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed6730-a27e-4f0c-b16d-b7758dad7192 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.014267 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed6730-a27e-4f0c-b16d-b7758dad7192 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.014753 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed6730-a27e-4f0c-b16d-b7758dad7192 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.016879 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed6730-a27e-4f0c-b16d-b7758dad7192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:59.019156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4a305-5aef-4a10-845b-ce43edc6dcd6 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:59.019752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed6730-a27e-4f0c-b16d-b7758dad7192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:59.021939 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed6730-a27e-4f0c-b16d-b7758dad7192 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:59.048330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 299/600] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:59.054663 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed6730-a27e-4f0c-b16d-b7758dad7192 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:59.096271 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dec60c-fea2-4b39-807a-c042a59f0cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:59.097644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dec60c-fea2-4b39-807a-c042a59f0cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.098132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dec60c-fea2-4b39-807a-c042a59f0cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.098629 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dec60c-fea2-4b39-807a-c042a59f0cd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.099568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dec60c-fea2-4b39-807a-c042a59f0cd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:59.101375 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dec60c-fea2-4b39-807a-c042a59f0cd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:59.102959 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dec60c-fea2-4b39-807a-c042a59f0cd5 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:59.117333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:59.130878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 302/601] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:59.145731 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dec60c-fea2-4b39-807a-c042a59f0cd5 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:59.191436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed6730-a27e-4f0c-b16d-b7758dad7192 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:59.216964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:59.229506 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af583692-cc67-4b7f-a641-ea60a88772f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:59.230597 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af583692-cc67-4b7f-a641-ea60a88772f4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.231085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af583692-cc67-4b7f-a641-ea60a88772f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.231558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af583692-cc67-4b7f-a641-ea60a88772f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.238812 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af583692-cc67-4b7f-a641-ea60a88772f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:59.240691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af583692-cc67-4b7f-a641-ea60a88772f4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:59.255910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-af583692-cc67-4b7f-a641-ea60a88772f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:59.255910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:59.255910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:59.255910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:59.255910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:59.255910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:59.255910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:59.255910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:59.255910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:59.255910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:59.255910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:59.255910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:59.255910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:59.255910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:59.255910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:59.256870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:59.257740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:59.257740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:59.257740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:59.257740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:59.257740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:59.257740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:59.257740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:59.257740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:59.260228 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 300/602] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:59.273447 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af583692-cc67-4b7f-a641-ea60a88772f4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:59.325744 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dec60c-fea2-4b39-807a-c042a59f0cd5 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:59.380942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364320b2-d76d-4b5d-bbda-61f0ee1c42f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:59.381744 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364320b2-d76d-4b5d-bbda-61f0ee1c42f2 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.382026 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364320b2-d76d-4b5d-bbda-61f0ee1c42f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.382295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364320b2-d76d-4b5d-bbda-61f0ee1c42f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.393311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364320b2-d76d-4b5d-bbda-61f0ee1c42f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:59.394753 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364320b2-d76d-4b5d-bbda-61f0ee1c42f2 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:59.426144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-364320b2-d76d-4b5d-bbda-61f0ee1c42f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:59.426144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:59.426144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:59.426144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:59.426144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:59.426144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:59.426144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:59.426144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:59.426144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:59.426144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:59.426144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:59.426144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:59.426144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:59.426144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:59.426144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:59.427171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:59.428045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:59.428045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:59.428045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:59.428045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:59.428045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:59.428045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:59.428045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:59.428045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:10:59.428045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 303/603] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:59.434766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364320b2-d76d-4b5d-bbda-61f0ee1c42f2 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:59.491388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af583692-cc67-4b7f-a641-ea60a88772f4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:59.546156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:59.561081 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f2f59-0fd1-45c2-8bff-46576dbbb018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:59.561744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f2f59-0fd1-45c2-8bff-46576dbbb018 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.562022 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f2f59-0fd1-45c2-8bff-46576dbbb018 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.562294 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f2f59-0fd1-45c2-8bff-46576dbbb018 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.563090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f2f59-0fd1-45c2-8bff-46576dbbb018 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:59.563397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f2f59-0fd1-45c2-8bff-46576dbbb018 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:59.564859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f2f59-0fd1-45c2-8bff-46576dbbb018 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:59.578818 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 301/604] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:59.580614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10b214-0bca-4435-8d83-31fd02737b34 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.581130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10b214-0bca-4435-8d83-31fd02737b34 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.581608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10b214-0bca-4435-8d83-31fd02737b34 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.587810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 302/605] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:10:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:59.588821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10b214-0bca-4435-8d83-31fd02737b34 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:59.602203 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364320b2-d76d-4b5d-bbda-61f0ee1c42f2 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:59.636874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf6217-95be-492f-a743-1ff439bd9c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:59.637533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf6217-95be-492f-a743-1ff439bd9c95 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.637838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf6217-95be-492f-a743-1ff439bd9c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.638125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf6217-95be-492f-a743-1ff439bd9c95 None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.638917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf6217-95be-492f-a743-1ff439bd9c95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:59.639224 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf6217-95be-492f-a743-1ff439bd9c95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:59.640662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf6217-95be-492f-a743-1ff439bd9c95 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:59.653041 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 304/606] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:59.654275 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb3940-03bc-4ae9-bd04-60956ea74e1f None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.654561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb3940-03bc-4ae9-bd04-60956ea74e1f None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.654837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb3940-03bc-4ae9-bd04-60956ea74e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.660626 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:59.675536 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-5ccb3940-03bc-4ae9-bd04-60956ea74e1f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:59.676831 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5ccb3940-03bc-4ae9-bd04-60956ea74e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:59.705129 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10b214-0bca-4435-8d83-31fd02737b34 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:59.752324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 305/607] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:59.752324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb3940-03bc-4ae9-bd04-60956ea74e1f None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:59.762339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247ceed-0899-46bb-877a-87309c0d09be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:59.763286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247ceed-0899-46bb-877a-87309c0d09be None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.763571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247ceed-0899-46bb-877a-87309c0d09be None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.763848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247ceed-0899-46bb-877a-87309c0d09be None admin] PATH_INFO: `/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/74327a10eaab43e0af02050da1a24f62/roles/7fe62c5c771d41c28f790fade1df9357` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.784927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247ceed-0899-46bb-877a-87309c0d09be None admin] RBAC: Authorizing `identity:create_grant(project_id=5e2a50ebbb48417c911df499d98a0157, user_id=74327a10eaab43e0af02050da1a24f62, role_id=7fe62c5c771d41c28f790fade1df9357)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:59.787006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247ceed-0899-46bb-877a-87309c0d09be None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:59.803446 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 303/608] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:59 2022] PUT /identity/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/74327a10eaab43e0af02050da1a24f62/roles/7fe62c5c771d41c28f790fade1df9357 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:59.804512 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247ceed-0899-46bb-877a-87309c0d09be None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:59.824540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:59.882944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb3940-03bc-4ae9-bd04-60956ea74e1f None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:59.915059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247ceed-0899-46bb-877a-87309c0d09be None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:59.917124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:59.922455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c0b17-9482-4d83-a374-4c8683e57085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:59.923668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c0b17-9482-4d83-a374-4c8683e57085 None admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.924309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c0b17-9482-4d83-a374-4c8683e57085 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.924797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c0b17-9482-4d83-a374-4c8683e57085 None admin] PATH_INFO: `/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/69aef294bb4943c3822ad7d0a3d91fd5/roles/7fe62c5c771d41c28f790fade1df9357` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.941769 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c0b17-9482-4d83-a374-4c8683e57085 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e2a50ebbb48417c911df499d98a0157, user_id=69aef294bb4943c3822ad7d0a3d91fd5, role_id=7fe62c5c771d41c28f790fade1df9357)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:59.944099 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c0b17-9482-4d83-a374-4c8683e57085 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:59.954076 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb5de9-c999-4148-bfca-d70d2aa0871a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:59.954829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb5de9-c999-4148-bfca-d70d2aa0871a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.955118 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb5de9-c999-4148-bfca-d70d2aa0871a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.955382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb5de9-c999-4148-bfca-d70d2aa0871a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.962957 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 306/609] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:59 2022] PUT /identity/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/69aef294bb4943c3822ad7d0a3d91fd5/roles/7fe62c5c771d41c28f790fade1df9357 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:59.984798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb5de9-c999-4148-bfca-d70d2aa0871a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:59.993163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-88cb5de9-c999-4148-bfca-d70d2aa0871a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:59.993163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:59.993163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:59.993163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:59.993163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:59.993163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:59.993163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:59.993163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:59.993163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:59.993163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:59.993163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:59.993163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:59.993163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:59.993163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:59.993163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:59.993163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:59.994158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:59.994158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:59.994158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:59.994158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:59.994158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:59.994158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:59.994158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:59.994158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:59.994158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:59.994158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:59.994158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:59.994158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:59.994158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:59.994158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:59.994158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:59.994158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:59.994992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:59.994992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:59.994992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:59.994992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:59.994992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:59.994992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:59.994992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:59.994992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:59.994992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:59.994992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:59.994992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:59.994992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:10:59.994992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 304/610] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:00.002691 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c0b17-9482-4d83-a374-4c8683e57085 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:00.139582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:00.144793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c0b17-9482-4d83-a374-4c8683e57085 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:00.182384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42fe06-8716-4ceb-b66c-0a80b85bf5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:00.183308 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42fe06-8716-4ceb-b66c-0a80b85bf5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:00.183588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42fe06-8716-4ceb-b66c-0a80b85bf5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:00.183863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42fe06-8716-4ceb-b66c-0a80b85bf5c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:00.184663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42fe06-8716-4ceb-b66c-0a80b85bf5c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:00.184969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42fe06-8716-4ceb-b66c-0a80b85bf5c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:00.186732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42fe06-8716-4ceb-b66c-0a80b85bf5c4 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:00.197028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 307/611] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:11:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:00.202502 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb5de9-c999-4148-bfca-d70d2aa0871a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:00.315273 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb5de9-c999-4148-bfca-d70d2aa0871a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:00.357280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae61af-0bc3-44e2-b949-cfc8ac4c5855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:00.358191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae61af-0bc3-44e2-b949-cfc8ac4c5855 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:00.358477 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae61af-0bc3-44e2-b949-cfc8ac4c5855 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:00.358745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae61af-0bc3-44e2-b949-cfc8ac4c5855 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:00.374557 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae61af-0bc3-44e2-b949-cfc8ac4c5855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:00.376015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae61af-0bc3-44e2-b949-cfc8ac4c5855 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:00.404101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-58ae61af-0bc3-44e2-b949-cfc8ac4c5855 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:00.404101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:00.404101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:00.404101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:00.404101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:00.404101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:00.404101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:00.404101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:00.404101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:00.404101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:00.404101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:00.404101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:00.404101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:00.404101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:00.404101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:00.405287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:00.406600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:00.406600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:00.406600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:00.406600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:00.406600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 305/612] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:00.410784 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42fe06-8716-4ceb-b66c-0a80b85bf5c4 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:00.542402 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42fe06-8716-4ceb-b66c-0a80b85bf5c4 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:00.577204 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cab1fb-8e22-400e-b42f-d26b9a34dcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:00.577877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cab1fb-8e22-400e-b42f-d26b9a34dcdf None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:00.578162 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cab1fb-8e22-400e-b42f-d26b9a34dcdf None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:00.578438 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cab1fb-8e22-400e-b42f-d26b9a34dcdf None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:00.579162 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cab1fb-8e22-400e-b42f-d26b9a34dcdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:00.579472 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cab1fb-8e22-400e-b42f-d26b9a34dcdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:00.581027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cab1fb-8e22-400e-b42f-d26b9a34dcdf None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:00.593988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 308/613] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:00.599445 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae61af-0bc3-44e2-b949-cfc8ac4c5855 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:00.613912 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:00.704512 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae61af-0bc3-44e2-b949-cfc8ac4c5855 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:00.742445 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17713f-41ee-4de1-911d-baf01b460b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:00.743254 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17713f-41ee-4de1-911d-baf01b460b87 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:00.743541 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17713f-41ee-4de1-911d-baf01b460b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:00.743821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17713f-41ee-4de1-911d-baf01b460b87 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:00.752180 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17713f-41ee-4de1-911d-baf01b460b87 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:00.757741 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17713f-41ee-4de1-911d-baf01b460b87 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:00.778199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-2a17713f-41ee-4de1-911d-baf01b460b87 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:00.778199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:00.778199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:00.778199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:00.778199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:00.778199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:00.778199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:00.778199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:00.778199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:00.778199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:00.778199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:00.778199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:00.778199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:00.778199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:00.778199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:00.778199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:00.780481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:00.782806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:00.782806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:00.782806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:00.782806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:00.782806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:00.782806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:00.782806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:00.782806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:00.782806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:00.782806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:00.782806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 306/614] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jul 20 19:11:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:00.789560 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cab1fb-8e22-400e-b42f-d26b9a34dcdf None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:00.823397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:00.894942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee911585-a86a-4111-af8e-8baa16f010c7 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:00.895383 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee911585-a86a-4111-af8e-8baa16f010c7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:00.895726 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee911585-a86a-4111-af8e-8baa16f010c7 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:00.898197 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 307/615] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:00.905365 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6b3e4-6728-481f-9b67-e792fbe83d7a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:00.905774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6b3e4-6728-481f-9b67-e792fbe83d7a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:00.906174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6b3e4-6728-481f-9b67-e792fbe83d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:00.934507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a4d6b3e4-6728-481f-9b67-e792fbe83d7a None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:00.935709 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a4d6b3e4-6728-481f-9b67-e792fbe83d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:00.967832 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cab1fb-8e22-400e-b42f-d26b9a34dcdf None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:01.039715 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abb493-d677-41a0-bb03-d724f767b4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:01.040417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abb493-d677-41a0-bb03-d724f767b4bc None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.040702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abb493-d677-41a0-bb03-d724f767b4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.040976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abb493-d677-41a0-bb03-d724f767b4bc None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.041830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abb493-d677-41a0-bb03-d724f767b4bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:01.042138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abb493-d677-41a0-bb03-d724f767b4bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:01.043637 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abb493-d677-41a0-bb03-d724f767b4bc None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:01.064666 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:01.080768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 308/616] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:11:01.085299 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 309/617] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jul 20 19:11:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:01.090774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6b3e4-6728-481f-9b67-e792fbe83d7a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:01.119913 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abb493-d677-41a0-bb03-d724f767b4bc None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:01.230894 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:01.250223 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6b3e4-6728-481f-9b67-e792fbe83d7a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:01.262382 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abb493-d677-41a0-bb03-d724f767b4bc None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:01.294547 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbaf53b-9fc8-47c3-9a6a-17dde7a364aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:01.295532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbaf53b-9fc8-47c3-9a6a-17dde7a364aa None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.295825 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbaf53b-9fc8-47c3-9a6a-17dde7a364aa None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.296100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbaf53b-9fc8-47c3-9a6a-17dde7a364aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.306836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fc32a-e082-4d9a-add1-0a65193faa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:01.307937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fc32a-e082-4d9a-add1-0a65193faa75 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.308666 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fc32a-e082-4d9a-add1-0a65193faa75 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.309137 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fc32a-e082-4d9a-add1-0a65193faa75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.320749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3fc32a-e082-4d9a-add1-0a65193faa75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:01.328816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-ea3fc32a-e082-4d9a-add1-0a65193faa75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:11:01.328816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:01.328816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:01.328816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:01.328816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:01.328816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:01.328816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:01.328816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:01.328816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:01.328816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:01.328816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:01.328816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:01.328816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:01.328816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:01.328816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:01.328816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:01.329919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:01.329919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:01.329919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:01.329919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:01.329919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:01.329919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:01.329919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:01.329919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:01.329919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:01.329919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:01.329919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:01.329919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:01.329919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:01.329919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:01.329919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:01.329919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:01.330865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:01.330865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:01.330865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:01.330865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:01.330865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:01.330865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:01.330865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:01.330865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:01.330865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:01.330865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:01.330865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:11:01.330865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:01.338899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 310/618] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:11:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:01.340592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07181eab-ffc2-436e-8963-5962994e1b62 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.341091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07181eab-ffc2-436e-8963-5962994e1b62 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.341613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07181eab-ffc2-436e-8963-5962994e1b62 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.352952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 311/619] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:01.354029 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07181eab-ffc2-436e-8963-5962994e1b62 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:01.371204 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:01.381725 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbaf53b-9fc8-47c3-9a6a-17dde7a364aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:01.383941 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbaf53b-9fc8-47c3-9a6a-17dde7a364aa None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:01.463680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07181eab-ffc2-436e-8963-5962994e1b62 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:01.504476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6f67e-ed0f-4039-b2e5-540fb4d8015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:01.505604 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6f67e-ed0f-4039-b2e5-540fb4d8015f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.505889 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6f67e-ed0f-4039-b2e5-540fb4d8015f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.506165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6f67e-ed0f-4039-b2e5-540fb4d8015f None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.506912 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6f67e-ed0f-4039-b2e5-540fb4d8015f None admin] RBAC: Adding query filter params (name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:01.507233 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6f67e-ed0f-4039-b2e5-540fb4d8015f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:01.508599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6f67e-ed0f-4039-b2e5-540fb4d8015f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:01.513079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 309/620] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:11:01.514454 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208c463-1e0c-4aea-9c35-9169902a31d8 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.514742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208c463-1e0c-4aea-9c35-9169902a31d8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.515017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208c463-1e0c-4aea-9c35-9169902a31d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.519234 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:01.526564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 312/621] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:11:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:01.527402 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6f67e-ed0f-4039-b2e5-540fb4d8015f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:01.541165 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4208c463-1e0c-4aea-9c35-9169902a31d8 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:01.542401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4208c463-1e0c-4aea-9c35-9169902a31d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:01.612601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 310/622] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:11:01.613443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208c463-1e0c-4aea-9c35-9169902a31d8 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:01.639152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6f67e-ed0f-4039-b2e5-540fb4d8015f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:01.640596 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:01.680073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3a405-8497-4c2f-8d15-a528abee82cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:01.681277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3a405-8497-4c2f-8d15-a528abee82cc None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.681725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3a405-8497-4c2f-8d15-a528abee82cc None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.682140 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3a405-8497-4c2f-8d15-a528abee82cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.693989 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3a405-8497-4c2f-8d15-a528abee82cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:01.695628 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3a405-8497-4c2f-8d15-a528abee82cc None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:01.714773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-fde3a405-8497-4c2f-8d15-a528abee82cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:01.714773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:01.714773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:01.714773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:01.714773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:01.714773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:01.714773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:01.714773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:01.714773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:01.714773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:01.714773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:01.714773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:01.714773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:01.714773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:01.714773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:01.715916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:01.717057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:01.717057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:01.717057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:01.717057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:01.717314 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208c463-1e0c-4aea-9c35-9169902a31d8 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:01.723398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 313/623] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:01.724996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3a405-8497-4c2f-8d15-a528abee82cc None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:01.751565 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4f121-3542-456a-93a5-497413ff9723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:01.752448 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4f121-3542-456a-93a5-497413ff9723 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.752783 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4f121-3542-456a-93a5-497413ff9723 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.753000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4f121-3542-456a-93a5-497413ff9723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.764142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4f121-3542-456a-93a5-497413ff9723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:01.765703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4f121-3542-456a-93a5-497413ff9723 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:01.784725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:01.784991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-e3f4f121-3542-456a-93a5-497413ff9723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:01.784991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:01.784991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:01.784991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:01.784991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:01.784991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:01.784991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:01.784991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:01.784991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:01.784991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:01.784991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:01.784991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:01.784991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:01.784991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:01.784991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:01.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:01.787140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:01.787140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:01.787140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:01.787140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:01.790969 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 311/624] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:01.792051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4f121-3542-456a-93a5-497413ff9723 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:01.841447 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3a405-8497-4c2f-8d15-a528abee82cc None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:01.872270 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:01.880462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72d8a9-b104-4fa5-90f1-df6161d1d6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:01.881497 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72d8a9-b104-4fa5-90f1-df6161d1d6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.882698 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72d8a9-b104-4fa5-90f1-df6161d1d6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.883126 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72d8a9-b104-4fa5-90f1-df6161d1d6e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.894219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4f121-3542-456a-93a5-497413ff9723 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:01.894685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72d8a9-b104-4fa5-90f1-df6161d1d6e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:01.903538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-5b72d8a9-b104-4fa5-90f1-df6161d1d6e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:11:01.903538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:01.903538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:01.903538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:01.903538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:01.903538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:01.903538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:01.903538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:01.903538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:01.903538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:01.903538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:01.903538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:01.903538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:01.903538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:01.903538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:01.903538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:01.904621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:01.904621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:01.904621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:01.904621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:01.904621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:01.904621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:01.904621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:01.904621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:01.904621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:01.904621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:01.904621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:01.904621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:01.904621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:01.904621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:01.904621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:01.904621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:01.905631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:01.905631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:01.905631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:01.905631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:01.905631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:01.905631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:01.905631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:01.905631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:01.905631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:01.905631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:01.905631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:11:01.905631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:01.912640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 314/625] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:11:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:01.913499 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72d8a9-b104-4fa5-90f1-df6161d1d6e9 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:01.937130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657ac60-2564-4aa9-a0f3-0efcbdb29413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:01.938739 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657ac60-2564-4aa9-a0f3-0efcbdb29413 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.939522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657ac60-2564-4aa9-a0f3-0efcbdb29413 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.940016 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657ac60-2564-4aa9-a0f3-0efcbdb29413 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.941649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657ac60-2564-4aa9-a0f3-0efcbdb29413 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:01.942183 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657ac60-2564-4aa9-a0f3-0efcbdb29413 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:01.944422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657ac60-2564-4aa9-a0f3-0efcbdb29413 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:01.963526 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 312/626] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:01.964456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657ac60-2564-4aa9-a0f3-0efcbdb29413 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:02.031332 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72d8a9-b104-4fa5-90f1-df6161d1d6e9 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:02.045435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:02.071476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8ef16-ac97-4ce3-8e0a-c5f4b8f7c25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:02.072578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8ef16-ac97-4ce3-8e0a-c5f4b8f7c25b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.073086 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8ef16-ac97-4ce3-8e0a-c5f4b8f7c25b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.073578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8ef16-ac97-4ce3-8e0a-c5f4b8f7c25b None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.074530 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8ef16-ac97-4ce3-8e0a-c5f4b8f7c25b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:02.075055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8ef16-ac97-4ce3-8e0a-c5f4b8f7c25b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:02.077033 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8ef16-ac97-4ce3-8e0a-c5f4b8f7c25b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:02.088073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657ac60-2564-4aa9-a0f3-0efcbdb29413 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:02.095367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 315/627] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:02.096582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8ef16-ac97-4ce3-8e0a-c5f4b8f7c25b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:02.129093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736b3a3-2800-40de-8ec0-a6105ef322e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:02.129765 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736b3a3-2800-40de-8ec0-a6105ef322e0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.130615 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736b3a3-2800-40de-8ec0-a6105ef322e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.131181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736b3a3-2800-40de-8ec0-a6105ef322e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.132210 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736b3a3-2800-40de-8ec0-a6105ef322e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:02.132626 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736b3a3-2800-40de-8ec0-a6105ef322e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:02.134079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736b3a3-2800-40de-8ec0-a6105ef322e0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:02.140674 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:02.145265 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 313/628] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:11:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:02.146878 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736b3a3-2800-40de-8ec0-a6105ef322e0 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:02.217312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8ef16-ac97-4ce3-8e0a-c5f4b8f7c25b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:02.253937 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736b3a3-2800-40de-8ec0-a6105ef322e0 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:02.260537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1752a46-84f3-45ed-9281-9f3923d04bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:02.262218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1752a46-84f3-45ed-9281-9f3923d04bef None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.262649 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1752a46-84f3-45ed-9281-9f3923d04bef None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.263559 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1752a46-84f3-45ed-9281-9f3923d04bef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.272531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1752a46-84f3-45ed-9281-9f3923d04bef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:02.274548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1752a46-84f3-45ed-9281-9f3923d04bef None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:02.288554 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4aee0d-da98-49ce-a526-1f16d8f5a70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:02.289370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4aee0d-da98-49ce-a526-1f16d8f5a70c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.289719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-d1752a46-84f3-45ed-9281-9f3923d04bef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:02.289719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:02.289719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:02.289719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:02.289719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:02.289719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:02.289719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:02.289719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:02.289719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:02.289719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:02.289719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:02.289719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:02.289719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:02.289719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:02.289719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:02.290590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:02.291460 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:02.291460 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:02.291460 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:02.291460 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:02.291460 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:02.291460 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:02.291460 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:02.291460 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:02.292545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4aee0d-da98-49ce-a526-1f16d8f5a70c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.292545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4aee0d-da98-49ce-a526-1f16d8f5a70c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.292713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 316/629] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:02.292713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1752a46-84f3-45ed-9281-9f3923d04bef None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:02.304420 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4aee0d-da98-49ce-a526-1f16d8f5a70c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:02.305870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4aee0d-da98-49ce-a526-1f16d8f5a70c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:02.321439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-8a4aee0d-da98-49ce-a526-1f16d8f5a70c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:11:02.321439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:02.321439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:02.321439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:02.321439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:02.321439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:02.321439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:02.321439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:02.321439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:02.321439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:02.321439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:02.321439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:02.321439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:02.321439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:02.321439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:02.321439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:02.322595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:02.323527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:02.323527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:02.323527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:02.323527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:02.323527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:02.323527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:02.323527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:02.323527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:02.323527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:11:02.323527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:02.323527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 314/630] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:02.325850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315778f2-7601-4f4c-b637-c537fe22a0ff None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.325850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315778f2-7601-4f4c-b637-c537fe22a0ff None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.325850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315778f2-7601-4f4c-b637-c537fe22a0ff None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.329735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 315/631] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:02.330785 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315778f2-7601-4f4c-b637-c537fe22a0ff None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:02.336214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:02.400990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:02.411851 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1752a46-84f3-45ed-9281-9f3923d04bef None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:02.450544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315778f2-7601-4f4c-b637-c537fe22a0ff None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:02.451696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45774b70-098c-409f-8fd9-957c7763e9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:02.452478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45774b70-098c-409f-8fd9-957c7763e9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.452760 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45774b70-098c-409f-8fd9-957c7763e9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.453037 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45774b70-098c-409f-8fd9-957c7763e9e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.464045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45774b70-098c-409f-8fd9-957c7763e9e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:02.465510 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45774b70-098c-409f-8fd9-957c7763e9e3 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:02.482869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-45774b70-098c-409f-8fd9-957c7763e9e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:02.482869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:02.482869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:02.482869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:02.482869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:02.482869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:02.482869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:02.482869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:02.482869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:02.482869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:02.482869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:02.482869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:02.482869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:02.482869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:02.482869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:02.483833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:02.485053 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:02.485053 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:02.485053 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:02.485053 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:02.485053 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 317/632] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:02.485053 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45774b70-098c-409f-8fd9-957c7763e9e3 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:02.492981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb84f4-f7c6-4cbe-a8ad-92f51a5b74af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:02.493898 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb84f4-f7c6-4cbe-a8ad-92f51a5b74af None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.494378 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb84f4-f7c6-4cbe-a8ad-92f51a5b74af None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.494832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb84f4-f7c6-4cbe-a8ad-92f51a5b74af None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.495807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb84f4-f7c6-4cbe-a8ad-92f51a5b74af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:02.496329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb84f4-f7c6-4cbe-a8ad-92f51a5b74af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:02.497935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb84f4-f7c6-4cbe-a8ad-92f51a5b74af None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:02.511804 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 316/633] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:02.513316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc959b5-4c41-4308-b6ab-9f39c9db3232 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.513601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc959b5-4c41-4308-b6ab-9f39c9db3232 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.513879 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc959b5-4c41-4308-b6ab-9f39c9db3232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.530924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-bfc959b5-4c41-4308-b6ab-9f39c9db3232 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:02.532097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bfc959b5-4c41-4308-b6ab-9f39c9db3232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:02.580787 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:02.597408 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45774b70-098c-409f-8fd9-957c7763e9e3 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:02.599047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 317/634] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:11:02.599873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc959b5-4c41-4308-b6ab-9f39c9db3232 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:02.631913 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833113c3-8821-4b64-8ac0-ea7cba1e095d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:02.632807 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833113c3-8821-4b64-8ac0-ea7cba1e095d None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.633211 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833113c3-8821-4b64-8ac0-ea7cba1e095d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.633663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833113c3-8821-4b64-8ac0-ea7cba1e095d None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.634581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833113c3-8821-4b64-8ac0-ea7cba1e095d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:02.634976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833113c3-8821-4b64-8ac0-ea7cba1e095d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:02.636765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833113c3-8821-4b64-8ac0-ea7cba1e095d None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:02.662242 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 318/635] 10.222.0.43 () {56 vars in 1135 bytes} [Wed Jul 20 19:11:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:02.663564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833113c3-8821-4b64-8ac0-ea7cba1e095d None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:02.671926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:02.722425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc959b5-4c41-4308-b6ab-9f39c9db3232 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:02.757092 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16db8e2-47c8-40c2-b244-7fc841e66fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:02.757744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16db8e2-47c8-40c2-b244-7fc841e66fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.758023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16db8e2-47c8-40c2-b244-7fc841e66fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.758291 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16db8e2-47c8-40c2-b244-7fc841e66fa2 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.759079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16db8e2-47c8-40c2-b244-7fc841e66fa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:02.759384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16db8e2-47c8-40c2-b244-7fc841e66fa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:02.760791 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16db8e2-47c8-40c2-b244-7fc841e66fa2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:02.767494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833113c3-8821-4b64-8ac0-ea7cba1e095d None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:02.773931 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 318/636] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:02.775240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36066ce2-fc07-4458-bb4d-a87d9f7f071e None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.775527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36066ce2-fc07-4458-bb4d-a87d9f7f071e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.775800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36066ce2-fc07-4458-bb4d-a87d9f7f071e None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.777051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 319/637] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:02.777728 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36066ce2-fc07-4458-bb4d-a87d9f7f071e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:02.805775 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cee590-c6f0-43d7-942a-3e0aa5a57bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:02.806430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cee590-c6f0-43d7-942a-3e0aa5a57bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.806716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cee590-c6f0-43d7-942a-3e0aa5a57bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.806992 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cee590-c6f0-43d7-942a-3e0aa5a57bd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.807821 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cee590-c6f0-43d7-942a-3e0aa5a57bd5 None admin] RBAC: Adding query filter params (role.id=7fe62c5c771d41c28f790fade1df9357, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=267b178236e046e4bc694b2da6f86eb0) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:02.808411 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cee590-c6f0-43d7-942a-3e0aa5a57bd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:02.809832 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cee590-c6f0-43d7-942a-3e0aa5a57bd5 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:02.815488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 319/638] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:02 2022] GET /identity/v3/role_assignments?user.id=267b178236e046e4bc694b2da6f86eb0&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=7fe62c5c771d41c28f790fade1df9357 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:02.816261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cee590-c6f0-43d7-942a-3e0aa5a57bd5 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:02.825528 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:02.894844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36066ce2-fc07-4458-bb4d-a87d9f7f071e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:02.907280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:02.929867 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cee590-c6f0-43d7-942a-3e0aa5a57bd5 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:02.934489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef68eed-9656-419e-a9c8-8a45e8a6b46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:02.935313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef68eed-9656-419e-a9c8-8a45e8a6b46b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.935580 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef68eed-9656-419e-a9c8-8a45e8a6b46b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.935854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef68eed-9656-419e-a9c8-8a45e8a6b46b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.948137 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef68eed-9656-419e-a9c8-8a45e8a6b46b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:02.956249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-fef68eed-9656-419e-a9c8-8a45e8a6b46b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:11:02.956249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:02.956249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:02.956249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:02.956249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:02.956249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:02.956249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:02.956249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:02.956249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:02.956249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:02.956249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:02.956249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:02.956249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:02.956249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:02.956249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:02.956249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:02.957282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:02.957282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:02.957282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:02.957282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:02.957282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:02.957282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:02.957282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:02.957282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:02.957282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:02.957282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:02.957282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:02.957282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:02.957282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:02.957282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:02.957282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:02.957282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:02.958316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:02.958316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:02.958316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:02.958316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:02.958316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:02.958316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:02.958316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:02.958316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:02.958316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:02.958316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:02.958316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:11:02.958316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:02.958316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 320/639] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:11:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:02.959024 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef68eed-9656-419e-a9c8-8a45e8a6b46b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:02.971868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7e8c3-2e90-444a-a094-1b87684e1f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:02.973115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7e8c3-2e90-444a-a094-1b87684e1f2e None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.973115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7e8c3-2e90-444a-a094-1b87684e1f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.973375 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7e8c3-2e90-444a-a094-1b87684e1f2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.986253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7e8c3-2e90-444a-a094-1b87684e1f2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:02.987703 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7e8c3-2e90-444a-a094-1b87684e1f2e None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:03.004877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-10d7e8c3-2e90-444a-a094-1b87684e1f2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:03.004877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:03.004877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:03.004877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:03.004877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:03.004877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:03.004877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:03.004877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:03.004877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:03.004877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:03.004877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:03.004877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:03.004877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:03.004877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:03.004877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:03.006301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:03.007165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:03.007165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:03.007165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:03.007165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:03.007165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 320/640] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:03.008644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf81d37-b12c-4706-94b2-4dab144c7fbb None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.008929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf81d37-b12c-4706-94b2-4dab144c7fbb None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.009200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf81d37-b12c-4706-94b2-4dab144c7fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.027897 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-caf81d37-b12c-4706-94b2-4dab144c7fbb None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:03.029147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-caf81d37-b12c-4706-94b2-4dab144c7fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:03.086188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:03.096073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef68eed-9656-419e-a9c8-8a45e8a6b46b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:03.102156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 321/641] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:11:03.102794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf81d37-b12c-4706-94b2-4dab144c7fbb None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:03.140325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb717009-9653-49f7-80e6-58dc8247aecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:03.141188 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb717009-9653-49f7-80e6-58dc8247aecb None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.141474 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb717009-9653-49f7-80e6-58dc8247aecb None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.141752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb717009-9653-49f7-80e6-58dc8247aecb None admin] PATH_INFO: `/v3/users/glance` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.149829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb717009-9653-49f7-80e6-58dc8247aecb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:03.151428 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb717009-9653-49f7-80e6-58dc8247aecb None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:03.167947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-bb717009-9653-49f7-80e6-58dc8247aecb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:11:03.167947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:03.167947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:03.167947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:03.167947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:03.167947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:03.167947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:03.167947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:03.167947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:03.167947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:03.167947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:03.167947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:03.167947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:03.167947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:03.167947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:03.167947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:03.169008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:03.170138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:03.170138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:03.170138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:03.170138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:03.170138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:03.170138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:03.170138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:03.170138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:03.170138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:11:03.170138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:03.170138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 321/642] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:03.170751 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb717009-9653-49f7-80e6-58dc8247aecb None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:03.206754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:03.219928 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf81d37-b12c-4706-94b2-4dab144c7fbb None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:03.253278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c37d2f-3a8c-48fb-8d8f-a0f8bc0dbf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:03.253944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c37d2f-3a8c-48fb-8d8f-a0f8bc0dbf7e None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.254228 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c37d2f-3a8c-48fb-8d8f-a0f8bc0dbf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.254502 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c37d2f-3a8c-48fb-8d8f-a0f8bc0dbf7e None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.255758 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c37d2f-3a8c-48fb-8d8f-a0f8bc0dbf7e None admin] RBAC: Adding query filter params (name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:03.256063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c37d2f-3a8c-48fb-8d8f-a0f8bc0dbf7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:03.257463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c37d2f-3a8c-48fb-8d8f-a0f8bc0dbf7e None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:03.265420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 322/643] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:11:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:03.266216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c37d2f-3a8c-48fb-8d8f-a0f8bc0dbf7e None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:03.287578 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb717009-9653-49f7-80e6-58dc8247aecb None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:03.319606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:03.327693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c955e-1ba2-411e-9a47-e32d5dadd213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:03.328405 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c955e-1ba2-411e-9a47-e32d5dadd213 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.328763 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c955e-1ba2-411e-9a47-e32d5dadd213 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.329039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c955e-1ba2-411e-9a47-e32d5dadd213 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.329878 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c955e-1ba2-411e-9a47-e32d5dadd213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:03.330185 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c955e-1ba2-411e-9a47-e32d5dadd213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:03.331658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c955e-1ba2-411e-9a47-e32d5dadd213 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:03.354626 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 322/644] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:03.355289 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c955e-1ba2-411e-9a47-e32d5dadd213 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:03.377943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c37d2f-3a8c-48fb-8d8f-a0f8bc0dbf7e None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:03.414099 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b6b15-0fe4-4006-a99e-3eeaeea8b747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:03.414812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b6b15-0fe4-4006-a99e-3eeaeea8b747 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.415090 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b6b15-0fe4-4006-a99e-3eeaeea8b747 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.415350 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b6b15-0fe4-4006-a99e-3eeaeea8b747 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.425630 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b6b15-0fe4-4006-a99e-3eeaeea8b747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:03.434119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-630b6b15-0fe4-4006-a99e-3eeaeea8b747 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:11:03.434119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:03.434119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:03.434119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:03.434119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:03.434119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:03.434119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:03.434119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:03.434119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:03.434119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:03.434119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:03.434119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:03.434119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:03.434119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:03.434119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:03.434119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:03.435061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:03.435061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:03.435061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:03.435061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:03.435061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:03.435061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:03.435061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:03.435061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:03.435061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:03.435061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:03.435061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:03.435061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:03.435061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:03.435061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:03.435061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:03.435061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:03.436027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:03.436027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:03.436027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:03.436027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:03.436027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:03.436027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:03.436027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:03.436027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:03.436027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:03.436027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:03.436027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:11:03.436027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:03.436027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 323/645] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:11:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:03.437214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b6b15-0fe4-4006-a99e-3eeaeea8b747 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:03.454798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:03.466046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c955e-1ba2-411e-9a47-e32d5dadd213 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:03.501848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299259b-7936-4793-9f10-8adfaf6e760b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:03.502520 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299259b-7936-4793-9f10-8adfaf6e760b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.502809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299259b-7936-4793-9f10-8adfaf6e760b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.503081 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299259b-7936-4793-9f10-8adfaf6e760b None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.503927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299259b-7936-4793-9f10-8adfaf6e760b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:03.504239 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299259b-7936-4793-9f10-8adfaf6e760b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:03.505739 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299259b-7936-4793-9f10-8adfaf6e760b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:03.524223 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 323/646] 10.222.0.43 () {56 vars in 1137 bytes} [Wed Jul 20 19:11:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:03.525116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299259b-7936-4793-9f10-8adfaf6e760b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:03.543347 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b6b15-0fe4-4006-a99e-3eeaeea8b747 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:03.554166 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:03.583944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a68447-1073-41e5-a3fa-2425d9eedc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:03.584788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a68447-1073-41e5-a3fa-2425d9eedc10 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.585072 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a68447-1073-41e5-a3fa-2425d9eedc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.585344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a68447-1073-41e5-a3fa-2425d9eedc10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.596504 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a68447-1073-41e5-a3fa-2425d9eedc10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:03.598185 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a68447-1073-41e5-a3fa-2425d9eedc10 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:03.617149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-b2a68447-1073-41e5-a3fa-2425d9eedc10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:03.617149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:03.617149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:03.617149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:03.617149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:03.617149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:03.617149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:03.617149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:03.617149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:03.617149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:03.617149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:03.617149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:03.617149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:03.617149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:03.617149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:03.618286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:03.619435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:03.619435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:03.619435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:03.619435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:03.620233 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 324/647] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:03.621048 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a68447-1073-41e5-a3fa-2425d9eedc10 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:03.641364 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299259b-7936-4793-9f10-8adfaf6e760b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:03.677025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90e755-8c53-461e-8bc8-0d3bd612e083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:03.677730 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90e755-8c53-461e-8bc8-0d3bd612e083 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.678008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90e755-8c53-461e-8bc8-0d3bd612e083 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.678277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90e755-8c53-461e-8bc8-0d3bd612e083 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.685162 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90e755-8c53-461e-8bc8-0d3bd612e083 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:03.686708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90e755-8c53-461e-8bc8-0d3bd612e083 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:03.700933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-8f90e755-8c53-461e-8bc8-0d3bd612e083 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:03.700933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:03.700933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:03.700933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:03.700933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:03.700933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:03.700933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:03.700933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:03.700933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:03.700933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:03.700933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:03.700933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:03.700933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:03.700933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:03.700933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:03.702026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:03.707832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:03.707832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:03.707832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:03.707832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:03.707832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:03.707832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:03.707832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:03.707832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:03.707832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 324/648] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:03.707832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90e755-8c53-461e-8bc8-0d3bd612e083 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:03.717550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:03.736965 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a68447-1073-41e5-a3fa-2425d9eedc10 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:03.771452 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c641d72-4cee-4d99-bd7a-6ca7e64da235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:03.772207 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c641d72-4cee-4d99-bd7a-6ca7e64da235 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.772583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c641d72-4cee-4d99-bd7a-6ca7e64da235 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.772921 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c641d72-4cee-4d99-bd7a-6ca7e64da235 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.773725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c641d72-4cee-4d99-bd7a-6ca7e64da235 None admin] RBAC: Adding query filter params (name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:03.774102 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c641d72-4cee-4d99-bd7a-6ca7e64da235 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:03.775509 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c641d72-4cee-4d99-bd7a-6ca7e64da235 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:03.788782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 325/649] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:11:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:03.788782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c641d72-4cee-4d99-bd7a-6ca7e64da235 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:03.817236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:03.826831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90e755-8c53-461e-8bc8-0d3bd612e083 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:03.862756 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15041690-d90e-43d1-9810-501743ea44a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:03.863406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15041690-d90e-43d1-9810-501743ea44a5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.863690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15041690-d90e-43d1-9810-501743ea44a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.863965 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15041690-d90e-43d1-9810-501743ea44a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.908101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c641d72-4cee-4d99-bd7a-6ca7e64da235 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:03.936655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15041690-d90e-43d1-9810-501743ea44a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:03.938030 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15041690-d90e-43d1-9810-501743ea44a5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:03.948335 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:03.954433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785fc2a-d5b8-4864-96fd-cab142f5978f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:03.955107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785fc2a-d5b8-4864-96fd-cab142f5978f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.955398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785fc2a-d5b8-4864-96fd-cab142f5978f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.955665 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785fc2a-d5b8-4864-96fd-cab142f5978f None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.956568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785fc2a-d5b8-4864-96fd-cab142f5978f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:03.956817 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785fc2a-d5b8-4864-96fd-cab142f5978f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:03.958259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785fc2a-d5b8-4864-96fd-cab142f5978f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:03.972344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 326/650] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:03.972344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785fc2a-d5b8-4864-96fd-cab142f5978f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.049850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 325/651] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:11:04.050959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15041690-d90e-43d1-9810-501743ea44a5 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.071293 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:04.091903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785fc2a-d5b8-4864-96fd-cab142f5978f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.128574 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5902b21-5375-4b7c-b619-abadad1d855c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.129432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5902b21-5375-4b7c-b619-abadad1d855c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.129794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5902b21-5375-4b7c-b619-abadad1d855c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.130154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5902b21-5375-4b7c-b619-abadad1d855c None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.131129 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5902b21-5375-4b7c-b619-abadad1d855c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:04.131524 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5902b21-5375-4b7c-b619-abadad1d855c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.133171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5902b21-5375-4b7c-b619-abadad1d855c None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.146413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 327/652] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:04.147594 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5902b21-5375-4b7c-b619-abadad1d855c None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.155903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15041690-d90e-43d1-9810-501743ea44a5 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.165605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:04.190784 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d46e2-a46c-4497-91bf-ff9b42519c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.191496 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d46e2-a46c-4497-91bf-ff9b42519c2c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.191775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d46e2-a46c-4497-91bf-ff9b42519c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.192047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d46e2-a46c-4497-91bf-ff9b42519c2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.202159 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d46e2-a46c-4497-91bf-ff9b42519c2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.203590 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d46e2-a46c-4497-91bf-ff9b42519c2c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.221608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-7c8d46e2-a46c-4497-91bf-ff9b42519c2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:04.221608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:04.221608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:04.221608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:04.221608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:04.221608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:04.221608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:04.221608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:04.221608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:04.221608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:04.221608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:04.221608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:04.221608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:04.221608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:04.221608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:04.222497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:04.223460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:04.223460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:04.223460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:04.223460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:04.223460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 326/653] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:04.223805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d46e2-a46c-4497-91bf-ff9b42519c2c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.263283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5902b21-5375-4b7c-b619-abadad1d855c None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.297716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43e04e-fca5-496b-92b8-cced337900fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.298513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43e04e-fca5-496b-92b8-cced337900fa None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.298802 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43e04e-fca5-496b-92b8-cced337900fa None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.299073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43e04e-fca5-496b-92b8-cced337900fa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.307259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43e04e-fca5-496b-92b8-cced337900fa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.308999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43e04e-fca5-496b-92b8-cced337900fa None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.323790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-4f43e04e-fca5-496b-92b8-cced337900fa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:11:04.323790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:04.323790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:04.323790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:04.323790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:04.323790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:04.323790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:04.323790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:04.323790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:04.323790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:04.323790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:04.323790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:04.323790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:04.323790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:04.323790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:04.323790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:04.324766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:04.325782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:04.325782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:04.325782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:04.325782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:04.325782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:04.325782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:04.325782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:04.325782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:04.325782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:11:04.325782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:04.325782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 328/654] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:04.326460 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43e04e-fca5-496b-92b8-cced337900fa None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.336109 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d46e2-a46c-4497-91bf-ff9b42519c2c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.337534 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:04.381181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83170a-aa47-4af8-b060-6663f65a9e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.381973 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83170a-aa47-4af8-b060-6663f65a9e34 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.382256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83170a-aa47-4af8-b060-6663f65a9e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.382529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83170a-aa47-4af8-b060-6663f65a9e34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.393967 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83170a-aa47-4af8-b060-6663f65a9e34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.395402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83170a-aa47-4af8-b060-6663f65a9e34 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.414075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-1a83170a-aa47-4af8-b060-6663f65a9e34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:04.414075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:04.414075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:04.414075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:04.414075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:04.414075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:04.414075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:04.414075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:04.414075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:04.414075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:04.414075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:04.414075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:04.414075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:04.414075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:04.414075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:04.414982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:04.415860 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:04.415860 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:04.415860 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:04.415860 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:04.415860 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 327/655] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:04.416129 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83170a-aa47-4af8-b060-6663f65a9e34 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.432043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:04.443587 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43e04e-fca5-496b-92b8-cced337900fa None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.480733 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fa4a8-909f-4821-86ee-861b9b4a50ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.481918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fa4a8-909f-4821-86ee-861b9b4a50ef None admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.482246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fa4a8-909f-4821-86ee-861b9b4a50ef None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.482516 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fa4a8-909f-4821-86ee-861b9b4a50ef None admin] PATH_INFO: `/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/1c944eab8a8948749bb8375787216b09/roles/7fe62c5c771d41c28f790fade1df9357` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.497383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458fa4a8-909f-4821-86ee-861b9b4a50ef None admin] RBAC: Authorizing `identity:create_grant(project_id=5e2a50ebbb48417c911df499d98a0157, user_id=1c944eab8a8948749bb8375787216b09, role_id=7fe62c5c771d41c28f790fade1df9357)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.498914 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458fa4a8-909f-4821-86ee-861b9b4a50ef None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.512903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 329/656] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:11:04 2022] PUT /identity/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/1c944eab8a8948749bb8375787216b09/roles/7fe62c5c771d41c28f790fade1df9357 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:11:04.514090 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fa4a8-909f-4821-86ee-861b9b4a50ef None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.537131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83170a-aa47-4af8-b060-6663f65a9e34 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.587710 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca14b6-8b71-49ff-acad-45d9f0f65797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.588878 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca14b6-8b71-49ff-acad-45d9f0f65797 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.589313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca14b6-8b71-49ff-acad-45d9f0f65797 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.589793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca14b6-8b71-49ff-acad-45d9f0f65797 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.591305 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca14b6-8b71-49ff-acad-45d9f0f65797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:04.592120 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca14b6-8b71-49ff-acad-45d9f0f65797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.593958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca14b6-8b71-49ff-acad-45d9f0f65797 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.611116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 328/657] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:04.612716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:04.612989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca14b6-8b71-49ff-acad-45d9f0f65797 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.633137 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fa4a8-909f-4821-86ee-861b9b4a50ef None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.679181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737aa13f-1e90-408a-ad4d-09dec6ec3633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.679859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737aa13f-1e90-408a-ad4d-09dec6ec3633 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.680140 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737aa13f-1e90-408a-ad4d-09dec6ec3633 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.680450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737aa13f-1e90-408a-ad4d-09dec6ec3633 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.681534 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737aa13f-1e90-408a-ad4d-09dec6ec3633 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:04.681654 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737aa13f-1e90-408a-ad4d-09dec6ec3633 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.683262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737aa13f-1e90-408a-ad4d-09dec6ec3633 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.702582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 330/658] 10.222.0.43 () {56 vars in 1139 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:04.703085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:04.703354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737aa13f-1e90-408a-ad4d-09dec6ec3633 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.741534 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca14b6-8b71-49ff-acad-45d9f0f65797 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.779786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f44c0-8b5b-4e77-a821-11889f67a74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.780883 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f44c0-8b5b-4e77-a821-11889f67a74d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.781600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f44c0-8b5b-4e77-a821-11889f67a74d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.782235 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f44c0-8b5b-4e77-a821-11889f67a74d None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.784139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f44c0-8b5b-4e77-a821-11889f67a74d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:04.784679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f44c0-8b5b-4e77-a821-11889f67a74d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.786386 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f44c0-8b5b-4e77-a821-11889f67a74d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.804411 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 329/659] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:04.806308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f44c0-8b5b-4e77-a821-11889f67a74d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.814390 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737aa13f-1e90-408a-ad4d-09dec6ec3633 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.828992 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:04.852427 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc610b4-8066-45d2-bc61-31703bf3711a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.853227 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc610b4-8066-45d2-bc61-31703bf3711a None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.853508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc610b4-8066-45d2-bc61-31703bf3711a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.853780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc610b4-8066-45d2-bc61-31703bf3711a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.861393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc610b4-8066-45d2-bc61-31703bf3711a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.862942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc610b4-8066-45d2-bc61-31703bf3711a None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.878885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-2dc610b4-8066-45d2-bc61-31703bf3711a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:11:04.878885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:04.878885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:04.878885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:04.878885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:04.878885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:04.878885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:04.878885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:04.878885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:04.878885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:04.878885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:04.878885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:04.878885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:04.878885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:04.878885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:04.878885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:04.879940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:04.880937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:04.880937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:04.880937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:04.880937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:04.880937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:04.880937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:04.880937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:04.880937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:04.880937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:11:04.880937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:04.880937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 331/660] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:04.881534 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc610b4-8066-45d2-bc61-31703bf3711a None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.939671 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f44c0-8b5b-4e77-a821-11889f67a74d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.977240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:04.985198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59346db0-8179-4b17-8a9e-04c713e57e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.986232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59346db0-8179-4b17-8a9e-04c713e57e03 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.986637 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59346db0-8179-4b17-8a9e-04c713e57e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.987046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59346db0-8179-4b17-8a9e-04c713e57e03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.988591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc610b4-8066-45d2-bc61-31703bf3711a None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.024984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758d14b-f153-44f3-a0cb-27c16c756e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.025829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758d14b-f153-44f3-a0cb-27c16c756e91 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.026107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758d14b-f153-44f3-a0cb-27c16c756e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.026401 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758d14b-f153-44f3-a0cb-27c16c756e91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.033286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758d14b-f153-44f3-a0cb-27c16c756e91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.034720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758d14b-f153-44f3-a0cb-27c16c756e91 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.049291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-d758d14b-f153-44f3-a0cb-27c16c756e91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:05.049291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:05.049291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:05.049291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:05.049291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:05.049291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:05.049291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:05.049291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:05.049291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:05.049291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:05.049291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:05.049291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:05.049291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:05.049291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:05.049291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:05.050367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:05.051269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:05.051269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:05.051269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:05.051269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:05.051269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:05.051269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:05.051269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:05.051269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:05.051269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 332/661] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:05.051269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758d14b-f153-44f3-a0cb-27c16c756e91 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:05.067509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59346db0-8179-4b17-8a9e-04c713e57e03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.069007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59346db0-8179-4b17-8a9e-04c713e57e03 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.088846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:05.168251 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758d14b-f153-44f3-a0cb-27c16c756e91 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.176415 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 330/662] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:11:05.176415 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222776b-0a83-4bad-82a5-67e0dee70132 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.176415 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222776b-0a83-4bad-82a5-67e0dee70132 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.176415 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222776b-0a83-4bad-82a5-67e0dee70132 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.176415 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 331/663] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:05.176415 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222776b-0a83-4bad-82a5-67e0dee70132 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:05.210611 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:05.213732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875f11e-aaff-4f6b-8cc8-272f9a045f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.214424 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875f11e-aaff-4f6b-8cc8-272f9a045f79 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.214702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875f11e-aaff-4f6b-8cc8-272f9a045f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.214979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875f11e-aaff-4f6b-8cc8-272f9a045f79 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.215929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875f11e-aaff-4f6b-8cc8-272f9a045f79 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:05.216141 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875f11e-aaff-4f6b-8cc8-272f9a045f79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.217625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875f11e-aaff-4f6b-8cc8-272f9a045f79 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.235696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 333/664] 10.222.0.43 () {56 vars in 1137 bytes} [Wed Jul 20 19:11:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:05.240861 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875f11e-aaff-4f6b-8cc8-272f9a045f79 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:05.292196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222776b-0a83-4bad-82a5-67e0dee70132 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.328813 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf015f-15d1-49e7-b704-8877d644740b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.329477 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf015f-15d1-49e7-b704-8877d644740b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.329757 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf015f-15d1-49e7-b704-8877d644740b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.330031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf015f-15d1-49e7-b704-8877d644740b None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.330815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf015f-15d1-49e7-b704-8877d644740b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:05.331122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf015f-15d1-49e7-b704-8877d644740b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.332579 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf015f-15d1-49e7-b704-8877d644740b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.342039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:05.345167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 332/665] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:05.351689 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193153b6-f71d-4247-91d1-f98d158676a1 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.351991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193153b6-f71d-4247-91d1-f98d158676a1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.352268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193153b6-f71d-4247-91d1-f98d158676a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.357748 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875f11e-aaff-4f6b-8cc8-272f9a045f79 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.369577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-193153b6-f71d-4247-91d1-f98d158676a1 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:05.370878 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-193153b6-f71d-4247-91d1-f98d158676a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:05.395342 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d00b8e-914a-4a4f-a77f-a20497204862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.396132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d00b8e-914a-4a4f-a77f-a20497204862 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.396699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d00b8e-914a-4a4f-a77f-a20497204862 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.396699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d00b8e-914a-4a4f-a77f-a20497204862 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.407264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d00b8e-914a-4a4f-a77f-a20497204862 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.408885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d00b8e-914a-4a4f-a77f-a20497204862 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.427686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-75d00b8e-914a-4a4f-a77f-a20497204862 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:05.427686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:05.427686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:05.427686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:05.427686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:05.427686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:05.427686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:05.427686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:05.427686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:05.427686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:05.427686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:05.427686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:05.427686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:05.427686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:05.427686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:05.428800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:05.429700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:05.429700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:05.429700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:05.429700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:05.429700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 334/666] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:05.429700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d00b8e-914a-4a4f-a77f-a20497204862 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:05.452480 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 333/667] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:11:05.452480 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193153b6-f71d-4247-91d1-f98d158676a1 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:05.484804 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:05.540424 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d00b8e-914a-4a4f-a77f-a20497204862 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.566894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193153b6-f71d-4247-91d1-f98d158676a1 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.577146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aef06-63a8-419f-bad2-78264b1e4cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.577917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aef06-63a8-419f-bad2-78264b1e4cba None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.578206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aef06-63a8-419f-bad2-78264b1e4cba None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.578485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aef06-63a8-419f-bad2-78264b1e4cba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.591802 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:05.604137 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6297d76-0010-4aaf-9d64-2424ad72ea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.604820 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6297d76-0010-4aaf-9d64-2424ad72ea83 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.605104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6297d76-0010-4aaf-9d64-2424ad72ea83 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.605379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6297d76-0010-4aaf-9d64-2424ad72ea83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.606213 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6297d76-0010-4aaf-9d64-2424ad72ea83 None admin] RBAC: Adding query filter params (role.id=7fe62c5c771d41c28f790fade1df9357, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=d3e28557896e41c893f0de2590bc96d7) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:05.606524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6297d76-0010-4aaf-9d64-2424ad72ea83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.608074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6297d76-0010-4aaf-9d64-2424ad72ea83 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.613998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 334/668] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:05 2022] GET /identity/v3/role_assignments?user.id=d3e28557896e41c893f0de2590bc96d7&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=7fe62c5c771d41c28f790fade1df9357 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:05.613998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6297d76-0010-4aaf-9d64-2424ad72ea83 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:05.670489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7aef06-63a8-419f-bad2-78264b1e4cba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.672348 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7aef06-63a8-419f-bad2-78264b1e4cba None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.743048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:05.761873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6297d76-0010-4aaf-9d64-2424ad72ea83 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.807184 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25661d-8c9f-4c53-92ff-337328071cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.808453 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25661d-8c9f-4c53-92ff-337328071cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.808931 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25661d-8c9f-4c53-92ff-337328071cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.809818 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25661d-8c9f-4c53-92ff-337328071cf4 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.811003 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25661d-8c9f-4c53-92ff-337328071cf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:05.811502 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25661d-8c9f-4c53-92ff-337328071cf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.813803 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25661d-8c9f-4c53-92ff-337328071cf4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.826732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 335/669] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:11:05.828835 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 335/670] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:05.828949 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aef06-63a8-419f-bad2-78264b1e4cba None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:05.832029 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25661d-8c9f-4c53-92ff-337328071cf4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:05.884612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:05.956618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aef06-63a8-419f-bad2-78264b1e4cba None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.965802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25661d-8c9f-4c53-92ff-337328071cf4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:06.007605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f16962-8544-4fc5-88bc-c95b883489dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:06.008438 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f16962-8544-4fc5-88bc-c95b883489dd None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:06.008720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f16962-8544-4fc5-88bc-c95b883489dd None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:06.008989 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f16962-8544-4fc5-88bc-c95b883489dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:06.029814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b48b52-81b3-4008-9790-7b3f0b45a3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:06.030619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b48b52-81b3-4008-9790-7b3f0b45a3ed None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:06.030902 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b48b52-81b3-4008-9790-7b3f0b45a3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:06.031176 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b48b52-81b3-4008-9790-7b3f0b45a3ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:06.034853 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f16962-8544-4fc5-88bc-c95b883489dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:06.042656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-04f16962-8544-4fc5-88bc-c95b883489dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:06.042656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:06.042656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:06.042656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:06.042656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:06.042656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:06.042656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:06.042656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:06.042656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:06.042656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:06.042656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:06.042656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:06.042656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:06.042656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:06.042656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:06.042656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:06.043569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:06.043569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:06.043569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:06.043569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:06.043569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:06.043569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:06.043569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:06.043569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:06.043569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:06.043569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:06.043569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:06.043569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:06.043569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:06.043569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:06.043569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:06.043569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:06.044415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:06.044415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:06.044415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:06.044415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:06.044415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:06.044415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:06.044415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:06.044415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:06.044415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:06.044415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:06.044415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:06.044415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:06.052468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 336/671] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:06.053533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f16962-8544-4fc5-88bc-c95b883489dd None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:06.054731 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b48b52-81b3-4008-9790-7b3f0b45a3ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:06.058805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b48b52-81b3-4008-9790-7b3f0b45a3ed None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:06.078341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:06.090868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-77b48b52-81b3-4008-9790-7b3f0b45a3ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:06.090868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:06.090868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:06.090868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:06.090868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:06.090868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:06.090868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:06.090868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:06.090868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:06.090868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:06.090868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:06.090868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:06.090868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:06.090868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:06.090868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:06.091807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:06.093144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:06.093144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:06.093144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:06.093144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:06.097680 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 336/672] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:06.098587 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b48b52-81b3-4008-9790-7b3f0b45a3ed None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:06.226562 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b48b52-81b3-4008-9790-7b3f0b45a3ed None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:06.275049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15252622-3872-481c-9f2e-7f84fd511a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:06.276785 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15252622-3872-481c-9f2e-7f84fd511a49 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:06.277075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15252622-3872-481c-9f2e-7f84fd511a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:06.277346 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15252622-3872-481c-9f2e-7f84fd511a49 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:06.278101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15252622-3872-481c-9f2e-7f84fd511a49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:06.278405 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15252622-3872-481c-9f2e-7f84fd511a49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:06.279775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15252622-3872-481c-9f2e-7f84fd511a49 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:06.284354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f16962-8544-4fc5-88bc-c95b883489dd None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:06.287778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:06.294236 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 337/673] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:06.295159 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15252622-3872-481c-9f2e-7f84fd511a49 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:06.346916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ead044-7ef9-4950-a0c0-8d87305b7232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:06.349506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ead044-7ef9-4950-a0c0-8d87305b7232 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:06.350009 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ead044-7ef9-4950-a0c0-8d87305b7232 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:06.350485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ead044-7ef9-4950-a0c0-8d87305b7232 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:06.365201 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ead044-7ef9-4950-a0c0-8d87305b7232 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:06.366918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ead044-7ef9-4950-a0c0-8d87305b7232 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:06.384440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-a1ead044-7ef9-4950-a0c0-8d87305b7232 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:06.384440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:06.384440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:06.384440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:06.384440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:06.384440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:06.384440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:06.384440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:06.384440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:06.384440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:06.384440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:06.384440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:06.384440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:06.384440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:06.384440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:06.385361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:06.386226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:06.386226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:06.386226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:06.386226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:06.386226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:06.386226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:06.386226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:06.386226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:06.393199 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 337/674] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:06.394356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ead044-7ef9-4950-a0c0-8d87305b7232 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:06.409972 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:06.490771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15252622-3872-481c-9f2e-7f84fd511a49 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:06.534171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ead044-7ef9-4950-a0c0-8d87305b7232 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:06.564872 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0dda84-323e-47a9-b1ef-da20e44a7ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:06.565532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0dda84-323e-47a9-b1ef-da20e44a7ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:06.565812 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0dda84-323e-47a9-b1ef-da20e44a7ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:06.566143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0dda84-323e-47a9-b1ef-da20e44a7ef1 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:06.566802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0dda84-323e-47a9-b1ef-da20e44a7ef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:06.567108 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0dda84-323e-47a9-b1ef-da20e44a7ef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:06.568624 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0dda84-323e-47a9-b1ef-da20e44a7ef1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:06.581300 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a70ff3-8b7d-46b7-a320-3797d7b8fbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:06.582100 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a70ff3-8b7d-46b7-a320-3797d7b8fbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:06.582366 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a70ff3-8b7d-46b7-a320-3797d7b8fbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:06.582633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a70ff3-8b7d-46b7-a320-3797d7b8fbc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:06.585889 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 338/675] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:06.587282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0dda84-323e-47a9-b1ef-da20e44a7ef1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:06.598657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a70ff3-8b7d-46b7-a320-3797d7b8fbc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:06.600195 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a70ff3-8b7d-46b7-a320-3797d7b8fbc1 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:06.618779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-84a70ff3-8b7d-46b7-a320-3797d7b8fbc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:06.618779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:06.618779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:06.618779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:06.618779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:06.618779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:06.618779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:06.618779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:06.618779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:06.618779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:06.618779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:06.618779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:06.618779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:06.618779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:06.618779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:06.619716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:06.620605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:06.620605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:06.620605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:06.620605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:06.620605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 338/676] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:06.621312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a70ff3-8b7d-46b7-a320-3797d7b8fbc1 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:06.637311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:06.710238 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0dda84-323e-47a9-b1ef-da20e44a7ef1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:06.732738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:06.736454 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a70ff3-8b7d-46b7-a320-3797d7b8fbc1 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:06.749619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586fc7e-4960-4abf-bf77-bf6d33c4844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:06.750425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586fc7e-4960-4abf-bf77-bf6d33c4844e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:06.750817 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586fc7e-4960-4abf-bf77-bf6d33c4844e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:06.751090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586fc7e-4960-4abf-bf77-bf6d33c4844e None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:06.751997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586fc7e-4960-4abf-bf77-bf6d33c4844e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:06.752320 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586fc7e-4960-4abf-bf77-bf6d33c4844e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:06.753747 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586fc7e-4960-4abf-bf77-bf6d33c4844e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:06.770712 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 339/677] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:06.770712 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586fc7e-4960-4abf-bf77-bf6d33c4844e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:06.776642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c65b44-ee01-4c43-bc43-321bebccdd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:06.777516 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c65b44-ee01-4c43-bc43-321bebccdd66 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:06.777875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c65b44-ee01-4c43-bc43-321bebccdd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:06.778222 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c65b44-ee01-4c43-bc43-321bebccdd66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:06.785187 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c65b44-ee01-4c43-bc43-321bebccdd66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:06.787185 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c65b44-ee01-4c43-bc43-321bebccdd66 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:06.800909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-77c65b44-ee01-4c43-bc43-321bebccdd66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:06.800909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:06.800909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:06.800909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:06.800909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:06.800909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:06.800909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:06.800909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:06.800909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:06.800909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:06.800909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:06.800909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:06.800909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:06.800909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:06.800909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:06.802063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:06.803676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:06.803676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:06.803676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:06.803676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:06.803676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:06.803676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:06.803676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:06.803676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:06.804523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 339/678] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:06.805738 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1deb7-2311-496b-9630-f8e602e3c765 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:06.806027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1deb7-2311-496b-9630-f8e602e3c765 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:06.806307 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1deb7-2311-496b-9630-f8e602e3c765 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:06.812432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 340/679] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:06.812432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1deb7-2311-496b-9630-f8e602e3c765 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:06.885906 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:06.888003 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586fc7e-4960-4abf-bf77-bf6d33c4844e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:06.913805 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1deb7-2311-496b-9630-f8e602e3c765 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:06.931115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6692ec-6435-4a1e-a647-c56c8ba6efc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:06.931799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6692ec-6435-4a1e-a647-c56c8ba6efc5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:06.932246 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6692ec-6435-4a1e-a647-c56c8ba6efc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:06.932554 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6692ec-6435-4a1e-a647-c56c8ba6efc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:06.933927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6692ec-6435-4a1e-a647-c56c8ba6efc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:06.934242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6692ec-6435-4a1e-a647-c56c8ba6efc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:06.935976 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6692ec-6435-4a1e-a647-c56c8ba6efc5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:06.953910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 340/680] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:06.953910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6692ec-6435-4a1e-a647-c56c8ba6efc5 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:06.958632 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e814a2b-f4ce-404c-a885-0b500c3baf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:06.959293 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e814a2b-f4ce-404c-a885-0b500c3baf65 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:06.959583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e814a2b-f4ce-404c-a885-0b500c3baf65 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:06.959862 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e814a2b-f4ce-404c-a885-0b500c3baf65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:06.960735 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e814a2b-f4ce-404c-a885-0b500c3baf65 None admin] RBAC: Adding query filter params (role.id=7fe62c5c771d41c28f790fade1df9357, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=74327a10eaab43e0af02050da1a24f62) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:06.961049 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e814a2b-f4ce-404c-a885-0b500c3baf65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:06.962493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e814a2b-f4ce-404c-a885-0b500c3baf65 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:06.968692 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 341/681] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:06 2022] GET /identity/v3/role_assignments?user.id=74327a10eaab43e0af02050da1a24f62&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=7fe62c5c771d41c28f790fade1df9357 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:06.969915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e8ef6-2df5-4ace-9f51-8cac4b714c63 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:06.970225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e8ef6-2df5-4ace-9f51-8cac4b714c63 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:06.970547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e8ef6-2df5-4ace-9f51-8cac4b714c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:06.990219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-460e8ef6-2df5-4ace-9f51-8cac4b714c63 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:06.991588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-460e8ef6-2df5-4ace-9f51-8cac4b714c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:07.008416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:07.076515 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 342/682] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:11:07.077472 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e8ef6-2df5-4ace-9f51-8cac4b714c63 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:07.097853 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6692ec-6435-4a1e-a647-c56c8ba6efc5 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:07.147597 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53940f0-71da-440e-85b6-3b8fa39aba59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:07.148279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53940f0-71da-440e-85b6-3b8fa39aba59 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:07.148672 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53940f0-71da-440e-85b6-3b8fa39aba59 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:07.148952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53940f0-71da-440e-85b6-3b8fa39aba59 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:07.149748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53940f0-71da-440e-85b6-3b8fa39aba59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:07.150057 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53940f0-71da-440e-85b6-3b8fa39aba59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:07.151481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53940f0-71da-440e-85b6-3b8fa39aba59 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:07.157605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:07.166121 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 341/683] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:07.167211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53940f0-71da-440e-85b6-3b8fa39aba59 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:07.198564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e8ef6-2df5-4ace-9f51-8cac4b714c63 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:07.242088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952f5bb-a0fa-4487-bde0-6846a3b13eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:07.243037 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952f5bb-a0fa-4487-bde0-6846a3b13eee None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:07.243441 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952f5bb-a0fa-4487-bde0-6846a3b13eee None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:07.243817 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952f5bb-a0fa-4487-bde0-6846a3b13eee None admin] PATH_INFO: `/v3/users/nova` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:07.253246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952f5bb-a0fa-4487-bde0-6846a3b13eee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:07.255237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952f5bb-a0fa-4487-bde0-6846a3b13eee None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:07.274623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-4952f5bb-a0fa-4487-bde0-6846a3b13eee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:07.274623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:07.274623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:07.274623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:07.274623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:07.274623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:07.274623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:07.274623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:07.274623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:07.274623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:07.274623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:07.274623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:07.274623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:07.274623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:07.274623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:07.274623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:07.275797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:07.277055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:07.277055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:07.277055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:07.277055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:07.277055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:07.277055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:07.277055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:07.277055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:07.277055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:07.277055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:07.277055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 343/684] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jul 20 19:11:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:07.277788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952f5bb-a0fa-4487-bde0-6846a3b13eee None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:07.283389 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53940f0-71da-440e-85b6-3b8fa39aba59 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:07.291175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:07.321306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2af232-abd9-4cf1-a669-3419178ab6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:07.322398 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2af232-abd9-4cf1-a669-3419178ab6de None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:07.322947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2af232-abd9-4cf1-a669-3419178ab6de None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:07.323343 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2af232-abd9-4cf1-a669-3419178ab6de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:07.336108 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2af232-abd9-4cf1-a669-3419178ab6de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:07.344569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-2d2af232-abd9-4cf1-a669-3419178ab6de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:11:07.344569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:07.344569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:07.344569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:07.344569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:07.344569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:07.344569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:07.344569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:07.344569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:07.344569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:07.344569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:07.344569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:07.344569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:07.344569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:07.344569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:07.344569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:07.345738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:07.345738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:07.345738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:07.345738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:07.345738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:07.345738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:07.345738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:07.345738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:07.345738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:07.345738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:07.345738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:07.345738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:07.345738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:07.345738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:07.345738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:07.345738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:07.346752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:07.346752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:07.346752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:07.346752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:07.346752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:07.346752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:07.346752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:07.346752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:07.346752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:07.346752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:07.346752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:11:07.346752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:07.348469 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 342/685] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:11:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:07.352311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2af232-abd9-4cf1-a669-3419178ab6de None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:07.396939 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952f5bb-a0fa-4487-bde0-6846a3b13eee None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:07.430395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:07.440278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656bf5c-fbc7-4f96-9e01-21a8f187e1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:07.441104 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656bf5c-fbc7-4f96-9e01-21a8f187e1bf None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:07.441488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656bf5c-fbc7-4f96-9e01-21a8f187e1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:07.441862 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656bf5c-fbc7-4f96-9e01-21a8f187e1bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:07.443325 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656bf5c-fbc7-4f96-9e01-21a8f187e1bf None admin] RBAC: Adding query filter params (role.id=7fe62c5c771d41c28f790fade1df9357, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=69aef294bb4943c3822ad7d0a3d91fd5) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:07.443739 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656bf5c-fbc7-4f96-9e01-21a8f187e1bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:07.445482 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656bf5c-fbc7-4f96-9e01-21a8f187e1bf None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:07.452188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 344/686] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:07 2022] GET /identity/v3/role_assignments?user.id=69aef294bb4943c3822ad7d0a3d91fd5&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=7fe62c5c771d41c28f790fade1df9357 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:07.453230 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656bf5c-fbc7-4f96-9e01-21a8f187e1bf None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:07.491633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2af232-abd9-4cf1-a669-3419178ab6de None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:07.557768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b3c4c-0b1a-4910-abb8-c3710bde65e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:07.558869 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b3c4c-0b1a-4910-abb8-c3710bde65e9 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:07.559256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b3c4c-0b1a-4910-abb8-c3710bde65e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:07.559631 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b3c4c-0b1a-4910-abb8-c3710bde65e9 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:07.561149 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b3c4c-0b1a-4910-abb8-c3710bde65e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:07.561149 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b3c4c-0b1a-4910-abb8-c3710bde65e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:07.562670 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b3c4c-0b1a-4910-abb8-c3710bde65e9 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:07.576461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656bf5c-fbc7-4f96-9e01-21a8f187e1bf None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:07.599621 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 343/687] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jul 20 19:11:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:07.604885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b3c4c-0b1a-4910-abb8-c3710bde65e9 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:07.619839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae6b71-d3d4-45d3-90c5-59a4e76c712c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:07.621452 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae6b71-d3d4-45d3-90c5-59a4e76c712c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:07.621926 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae6b71-d3d4-45d3-90c5-59a4e76c712c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:07.622382 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae6b71-d3d4-45d3-90c5-59a4e76c712c None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:07.623389 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae6b71-d3d4-45d3-90c5-59a4e76c712c None admin] RBAC: Adding query filter params (name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:07.623910 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae6b71-d3d4-45d3-90c5-59a4e76c712c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:07.625519 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae6b71-d3d4-45d3-90c5-59a4e76c712c None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:07.636869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 345/688] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:11:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:07.645347 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae6b71-d3d4-45d3-90c5-59a4e76c712c None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:07.645764 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:07.747732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:07.760926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b3c4c-0b1a-4910-abb8-c3710bde65e9 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:07.801504 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13de46-9fe7-4c21-a8d2-03c301744ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:07.802549 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13de46-9fe7-4c21-a8d2-03c301744ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:07.803051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13de46-9fe7-4c21-a8d2-03c301744ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:07.803537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13de46-9fe7-4c21-a8d2-03c301744ed5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:07.809681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae6b71-d3d4-45d3-90c5-59a4e76c712c None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:07.864180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36451237-29d2-417b-bfc2-4660fa55c5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:07.866016 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36451237-29d2-417b-bfc2-4660fa55c5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:07.866543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36451237-29d2-417b-bfc2-4660fa55c5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:07.867052 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36451237-29d2-417b-bfc2-4660fa55c5a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:07.886509 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36451237-29d2-417b-bfc2-4660fa55c5a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:07.887987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36451237-29d2-417b-bfc2-4660fa55c5a6 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:07.895087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13de46-9fe7-4c21-a8d2-03c301744ed5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:07.898703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13de46-9fe7-4c21-a8d2-03c301744ed5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:07.906815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-36451237-29d2-417b-bfc2-4660fa55c5a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:07.906815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:07.906815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:07.906815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:07.906815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:07.906815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:07.906815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:07.906815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:07.906815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:07.906815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:07.906815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:07.906815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:07.906815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:07.906815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:07.906815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:07.907752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:07.909136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:07.909136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:07.909136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:07.909136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:07.912397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 346/689] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:07.914442 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36451237-29d2-417b-bfc2-4660fa55c5a6 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:07.924710 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:08.019614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 344/690] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:11:08.021542 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36451237-29d2-417b-bfc2-4660fa55c5a6 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.022092 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f5755-19e3-4f90-8e43-d77da5435668 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.022576 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f5755-19e3-4f90-8e43-d77da5435668 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.023065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f5755-19e3-4f90-8e43-d77da5435668 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.025257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 345/691] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:08.026215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f5755-19e3-4f90-8e43-d77da5435668 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.029373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:08.060200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67335a-3e70-4708-9b76-5f906f874a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.060903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67335a-3e70-4708-9b76-5f906f874a54 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.061191 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67335a-3e70-4708-9b76-5f906f874a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.061471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67335a-3e70-4708-9b76-5f906f874a54 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.062212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67335a-3e70-4708-9b76-5f906f874a54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:08.062523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67335a-3e70-4708-9b76-5f906f874a54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:08.063977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67335a-3e70-4708-9b76-5f906f874a54 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:08.076679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 347/692] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:08.077955 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b2d48-57e7-46ae-a286-71e8788dfe77 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.078211 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b2d48-57e7-46ae-a286-71e8788dfe77 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.078567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b2d48-57e7-46ae-a286-71e8788dfe77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.099844 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-229b2d48-57e7-46ae-a286-71e8788dfe77 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:08.101079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-229b2d48-57e7-46ae-a286-71e8788dfe77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:08.140329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f5755-19e3-4f90-8e43-d77da5435668 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.160457 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:08.172875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 348/693] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:11:08.173824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b2d48-57e7-46ae-a286-71e8788dfe77 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.181111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facad221-f61e-4e83-b451-9c6b908cecdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.182199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facad221-f61e-4e83-b451-9c6b908cecdb None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.182678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facad221-f61e-4e83-b451-9c6b908cecdb None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.183415 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facad221-f61e-4e83-b451-9c6b908cecdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.195962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facad221-f61e-4e83-b451-9c6b908cecdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:08.197850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facad221-f61e-4e83-b451-9c6b908cecdb None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:08.216179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-facad221-f61e-4e83-b451-9c6b908cecdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:08.216179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:08.216179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:08.216179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:08.216179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:08.216179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:08.216179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:08.216179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:08.216179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:08.216179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:08.216179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:08.216179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:08.216179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:08.216179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:08.216179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:08.217459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:08.218336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:08.218336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:08.218336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:08.218336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:08.225336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 346/694] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:08.226467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facad221-f61e-4e83-b451-9c6b908cecdb None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.289257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:08.298884 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b2d48-57e7-46ae-a286-71e8788dfe77 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.338567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facad221-f61e-4e83-b451-9c6b908cecdb None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.353625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940f405-8108-480c-a5b4-826bb72e3ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.354981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940f405-8108-480c-a5b4-826bb72e3ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.355380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940f405-8108-480c-a5b4-826bb72e3ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.355768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940f405-8108-480c-a5b4-826bb72e3ab2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.364670 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940f405-8108-480c-a5b4-826bb72e3ab2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:08.366261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940f405-8108-480c-a5b4-826bb72e3ab2 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:08.387230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e93c08-ca1d-4b8e-9b9e-57811e3ac67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.388027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e93c08-ca1d-4b8e-9b9e-57811e3ac67c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.388339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e93c08-ca1d-4b8e-9b9e-57811e3ac67c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.388894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e93c08-ca1d-4b8e-9b9e-57811e3ac67c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.389754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-c940f405-8108-480c-a5b4-826bb72e3ab2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:11:08.389754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:08.389754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:08.389754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:08.389754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:08.389754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:08.389754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:08.389754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:08.389754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:08.389754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:08.389754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:08.389754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:08.389754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:08.389754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:08.389754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:08.389754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:08.390655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:08.391585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:08.391585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:08.391585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:08.391585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:08.391585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:08.391585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:08.391585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:08.391585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:08.391585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:11:08.391585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:08.392761 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 349/695] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:08.393220 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940f405-8108-480c-a5b4-826bb72e3ab2 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.405851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e93c08-ca1d-4b8e-9b9e-57811e3ac67c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:08.407341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e93c08-ca1d-4b8e-9b9e-57811e3ac67c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:08.431173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-15e93c08-ca1d-4b8e-9b9e-57811e3ac67c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:08.431173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:08.431173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:08.431173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:08.431173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:08.431173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:08.431173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:08.431173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:08.431173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:08.431173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:08.431173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:08.431173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:08.431173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:08.431173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:08.431173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:08.432050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:08.433957 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:08.433957 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:08.433957 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:08.433957 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:08.433957 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 347/696] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:08.433957 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e93c08-ca1d-4b8e-9b9e-57811e3ac67c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.461609 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:08.516541 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940f405-8108-480c-a5b4-826bb72e3ab2 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.554297 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e93c08-ca1d-4b8e-9b9e-57811e3ac67c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.555571 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73b8e1-c54a-4dc9-ba04-35f448d3595c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.556579 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73b8e1-c54a-4dc9-ba04-35f448d3595c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.557080 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73b8e1-c54a-4dc9-ba04-35f448d3595c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.557552 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73b8e1-c54a-4dc9-ba04-35f448d3595c None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.558725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73b8e1-c54a-4dc9-ba04-35f448d3595c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:08.559178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73b8e1-c54a-4dc9-ba04-35f448d3595c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:08.560712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73b8e1-c54a-4dc9-ba04-35f448d3595c None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:08.574337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 350/697] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:08.575553 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73b8e1-c54a-4dc9-ba04-35f448d3595c None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.591751 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a46394-3373-4db8-8e48-73ba6bec2232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.592477 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a46394-3373-4db8-8e48-73ba6bec2232 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.592770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a46394-3373-4db8-8e48-73ba6bec2232 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.593046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a46394-3373-4db8-8e48-73ba6bec2232 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.594205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a46394-3373-4db8-8e48-73ba6bec2232 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:08.594526 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a46394-3373-4db8-8e48-73ba6bec2232 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:08.595955 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a46394-3373-4db8-8e48-73ba6bec2232 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:08.597226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:08.621796 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 348/698] 10.222.0.43 () {56 vars in 1135 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:08.622676 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a46394-3373-4db8-8e48-73ba6bec2232 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.687422 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73b8e1-c54a-4dc9-ba04-35f448d3595c None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.709543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:08.727177 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d0036-d77a-49b4-b8bd-b0405060fb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.728711 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d0036-d77a-49b4-b8bd-b0405060fb9d None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.729331 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d0036-d77a-49b4-b8bd-b0405060fb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.729828 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d0036-d77a-49b4-b8bd-b0405060fb9d None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.730748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a46394-3373-4db8-8e48-73ba6bec2232 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.730949 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d0036-d77a-49b4-b8bd-b0405060fb9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:08.731448 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d0036-d77a-49b4-b8bd-b0405060fb9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:08.734269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d0036-d77a-49b4-b8bd-b0405060fb9d None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:08.750531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 351/699] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:08.752121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d0036-d77a-49b4-b8bd-b0405060fb9d None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.773588 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62099e2a-04d4-4210-80f3-404171739b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.774406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62099e2a-04d4-4210-80f3-404171739b59 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.774690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62099e2a-04d4-4210-80f3-404171739b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.774980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62099e2a-04d4-4210-80f3-404171739b59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.782456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62099e2a-04d4-4210-80f3-404171739b59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:08.783900 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62099e2a-04d4-4210-80f3-404171739b59 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:08.799097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-62099e2a-04d4-4210-80f3-404171739b59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:08.799097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:08.799097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:08.799097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:08.799097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:08.799097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:08.799097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:08.799097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:08.799097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:08.799097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:08.799097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:08.799097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:08.799097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:08.799097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:08.799097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:08.800025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:08.800943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:08.800943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:08.800943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:08.800943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:08.800943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:08.800943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:08.800943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:08.800943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:08.800943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 349/700] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:08.801426 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62099e2a-04d4-4210-80f3-404171739b59 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.860724 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:08.886643 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d0036-d77a-49b4-b8bd-b0405060fb9d None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.926292 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62099e2a-04d4-4210-80f3-404171739b59 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.932218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cba0cd-49a1-4f32-a61e-7a18417f697c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.933221 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cba0cd-49a1-4f32-a61e-7a18417f697c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.933557 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cba0cd-49a1-4f32-a61e-7a18417f697c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.933836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cba0cd-49a1-4f32-a61e-7a18417f697c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.967072 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc0d1f-75c1-45c1-bb42-184e23a63b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.967785 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc0d1f-75c1-45c1-bb42-184e23a63b41 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.968079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc0d1f-75c1-45c1-bb42-184e23a63b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.968392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc0d1f-75c1-45c1-bb42-184e23a63b41 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.969115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc0d1f-75c1-45c1-bb42-184e23a63b41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:08.970569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc0d1f-75c1-45c1-bb42-184e23a63b41 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:08.981218 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:09.000332 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 350/701] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:11:08 2022] POST /identity/v3/users => generated 254 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:09.000332 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc0d1f-75c1-45c1-bb42-184e23a63b41 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:09.066582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cba0cd-49a1-4f32-a61e-7a18417f697c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:09.068619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cba0cd-49a1-4f32-a61e-7a18417f697c None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:09.115505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc0d1f-75c1-45c1-bb42-184e23a63b41 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:09.155106 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57247bb9-6b0b-42cb-9d1e-bb9825f7f077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:09.155782 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57247bb9-6b0b-42cb-9d1e-bb9825f7f077 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:09.156069 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57247bb9-6b0b-42cb-9d1e-bb9825f7f077 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:09.156380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57247bb9-6b0b-42cb-9d1e-bb9825f7f077 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:09.157330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57247bb9-6b0b-42cb-9d1e-bb9825f7f077 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:09.157652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57247bb9-6b0b-42cb-9d1e-bb9825f7f077 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:09.159246 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57247bb9-6b0b-42cb-9d1e-bb9825f7f077 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:09.174007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 351/702] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:09.186266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57247bb9-6b0b-42cb-9d1e-bb9825f7f077 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:09.233532 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:09.257808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:09.266699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 352/703] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:11:09.276462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cba0cd-49a1-4f32-a61e-7a18417f697c None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:09.312034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57247bb9-6b0b-42cb-9d1e-bb9825f7f077 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:09.350579 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe142d-f907-4518-bdea-56fd22fb2172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:09.351249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe142d-f907-4518-bdea-56fd22fb2172 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:09.351536 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe142d-f907-4518-bdea-56fd22fb2172 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:09.351818 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe142d-f907-4518-bdea-56fd22fb2172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:09.352722 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe142d-f907-4518-bdea-56fd22fb2172 None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=267b178236e046e4bc694b2da6f86eb0) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:09.353820 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe142d-f907-4518-bdea-56fd22fb2172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:09.354613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe142d-f907-4518-bdea-56fd22fb2172 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:09.361727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 352/704] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:09 2022] GET /identity/v3/role_assignments?user.id=267b178236e046e4bc694b2da6f86eb0&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:09.364595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fb096-9964-49c7-93f5-1c8f1176c04a None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:09.364595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fb096-9964-49c7-93f5-1c8f1176c04a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:09.364595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fb096-9964-49c7-93f5-1c8f1176c04a None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:09.364973 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 353/705] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:09.375483 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd94c70-6ed0-48e1-a78b-0c3bc7a9bc05 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:09.375743 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd94c70-6ed0-48e1-a78b-0c3bc7a9bc05 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:09.376169 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd94c70-6ed0-48e1-a78b-0c3bc7a9bc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:09.396193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-dcd94c70-6ed0-48e1-a78b-0c3bc7a9bc05 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:09.397548 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-dcd94c70-6ed0-48e1-a78b-0c3bc7a9bc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:09.408170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cba0cd-49a1-4f32-a61e-7a18417f697c None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:09.481650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b68d1e-72a1-4268-adaa-0cb5a60470cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:09.482471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b68d1e-72a1-4268-adaa-0cb5a60470cb None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:09.482765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b68d1e-72a1-4268-adaa-0cb5a60470cb None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:09.483047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b68d1e-72a1-4268-adaa-0cb5a60470cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:09.505903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b68d1e-72a1-4268-adaa-0cb5a60470cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:09.507373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b68d1e-72a1-4268-adaa-0cb5a60470cb None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:09.528655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 354/706] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:11:09.537705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-23b68d1e-72a1-4268-adaa-0cb5a60470cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:09.537705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:09.537705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:09.537705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:09.537705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:09.537705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:09.537705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:09.537705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:09.537705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:09.537705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:09.537705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:09.537705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:09.537705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:09.537705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:09.537705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:09.538947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:09.539931 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:09.539931 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:09.539931 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:09.539931 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:09.541840 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 353/707] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:09.547058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd94c70-6ed0-48e1-a78b-0c3bc7a9bc05 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:09.554780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b68d1e-72a1-4268-adaa-0cb5a60470cb None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:09.599062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:09.636926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:09.725339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b68d1e-72a1-4268-adaa-0cb5a60470cb None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:09.730934 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd94c70-6ed0-48e1-a78b-0c3bc7a9bc05 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:09.787245 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b9560-d5db-4ad9-b62b-19f54c36f4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:09.788050 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b9560-d5db-4ad9-b62b-19f54c36f4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:09.788356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b9560-d5db-4ad9-b62b-19f54c36f4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:09.788643 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b9560-d5db-4ad9-b62b-19f54c36f4d8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:09.789309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b9560-d5db-4ad9-b62b-19f54c36f4d8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:09.790685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b9560-d5db-4ad9-b62b-19f54c36f4d8 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:09.796157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e45a9-d422-4593-8747-dd314bc7a8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:09.796852 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e45a9-d422-4593-8747-dd314bc7a8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:09.797135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e45a9-d422-4593-8747-dd314bc7a8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:09.797409 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e45a9-d422-4593-8747-dd314bc7a8e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:09.798147 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e45a9-d422-4593-8747-dd314bc7a8e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:09.798455 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e45a9-d422-4593-8747-dd314bc7a8e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:09.799817 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e45a9-d422-4593-8747-dd314bc7a8e7 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-337b9560-d5db-4ad9-b62b-19f54c36f4d8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:09.814797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:09.815899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:09.815899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:09.815899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:09.815899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:09.815899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:09.815899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:09.815899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:09.815899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:09.815899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:09.815899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:09.815899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:09.815899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:09.815899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:09.815899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:09.815899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:09.815899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:09.817952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:09.817952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:09.817952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:09.817952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:09.817952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:09.817952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:09.817952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:09.817952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:09.817952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:09.817952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:09.817952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:09.817952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:09.817952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:11:09.817952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:09.820405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 354/708] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:09 2022] GET /identity/v3/services/network => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:09.822590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b9560-d5db-4ad9-b62b-19f54c36f4d8 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:09.830846 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 355/709] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:09.837803 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e45a9-d422-4593-8747-dd314bc7a8e7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:10.004280 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:10.016437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e45a9-d422-4593-8747-dd314bc7a8e7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:10.016623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b9560-d5db-4ad9-b62b-19f54c36f4d8 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:10.044675 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:10.055127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8ba5c-14d5-4e8a-960c-8bc9777fd32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:10.056072 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8ba5c-14d5-4e8a-960c-8bc9777fd32a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.056526 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8ba5c-14d5-4e8a-960c-8bc9777fd32a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.056908 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8ba5c-14d5-4e8a-960c-8bc9777fd32a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.061092 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4be3f-b4bb-4c95-abfb-dec76e926366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:10.061767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4be3f-b4bb-4c95-abfb-dec76e926366 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.062051 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4be3f-b4bb-4c95-abfb-dec76e926366 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.062327 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4be3f-b4bb-4c95-abfb-dec76e926366 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.063043 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4be3f-b4bb-4c95-abfb-dec76e926366 None admin] RBAC: Adding query filter params (name=network) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:10.063349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4be3f-b4bb-4c95-abfb-dec76e926366 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:10.065032 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4be3f-b4bb-4c95-abfb-dec76e926366 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:10.065515 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8ba5c-14d5-4e8a-960c-8bc9777fd32a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:10.067201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8ba5c-14d5-4e8a-960c-8bc9777fd32a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:10.070712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 355/710] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:10.072094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780966c7-3141-461c-a195-3d91c0897c1f None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.072408 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780966c7-3141-461c-a195-3d91c0897c1f None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.072692 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780966c7-3141-461c-a195-3d91c0897c1f None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.078057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 356/711] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:10.079190 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780966c7-3141-461c-a195-3d91c0897c1f None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:10.088359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-b0a8ba5c-14d5-4e8a-960c-8bc9777fd32a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:10.088359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:10.088359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:10.088359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:10.088359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:10.088359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:10.088359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:10.088359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:10.088359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:10.088359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:10.088359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:10.088359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:10.088359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:10.088359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:10.088359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:10.089523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:10.090669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:10.090669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:10.090669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:10.090669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:10.090669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:10.090669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:10.090669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:10.090669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:10.091639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 356/712] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:10.097606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f509bb-94cf-4378-a41e-f5054274b76f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.097970 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f509bb-94cf-4378-a41e-f5054274b76f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.098330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f509bb-94cf-4378-a41e-f5054274b76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.117285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f6f509bb-94cf-4378-a41e-f5054274b76f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:10.118600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f6f509bb-94cf-4378-a41e-f5054274b76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:10.201820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780966c7-3141-461c-a195-3d91c0897c1f None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:10.204449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 357/713] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:11:10.205320 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f509bb-94cf-4378-a41e-f5054274b76f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:10.254261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09fef2-1038-439c-b007-842ebbd8889c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:10.255731 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09fef2-1038-439c-b007-842ebbd8889c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.255961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09fef2-1038-439c-b007-842ebbd8889c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.256243 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09fef2-1038-439c-b007-842ebbd8889c None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.257005 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09fef2-1038-439c-b007-842ebbd8889c None admin] RBAC: Adding query filter params (type=network) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:10.257312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09fef2-1038-439c-b007-842ebbd8889c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:10.258684 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09fef2-1038-439c-b007-842ebbd8889c None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:10.271275 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 357/714] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:10.272483 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09fef2-1038-439c-b007-842ebbd8889c None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:10.299450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:10.428678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:10.434799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f509bb-94cf-4378-a41e-f5054274b76f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:10.500943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09fef2-1038-439c-b007-842ebbd8889c None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:10.526864 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fea952-8871-4f5b-bf3c-6b77dc893fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:10.527903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fea952-8871-4f5b-bf3c-6b77dc893fbc None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.528578 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fea952-8871-4f5b-bf3c-6b77dc893fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.529171 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fea952-8871-4f5b-bf3c-6b77dc893fbc None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.530639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fea952-8871-4f5b-bf3c-6b77dc893fbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:10.531150 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fea952-8871-4f5b-bf3c-6b77dc893fbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:10.534161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fea952-8871-4f5b-bf3c-6b77dc893fbc None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:10.537595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e5909-721d-49c9-8ab4-4d61469349e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:10.538389 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e5909-721d-49c9-8ab4-4d61469349e5 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.538915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e5909-721d-49c9-8ab4-4d61469349e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.539085 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e5909-721d-49c9-8ab4-4d61469349e5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.539692 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888e5909-721d-49c9-8ab4-4d61469349e5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:10.541058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888e5909-721d-49c9-8ab4-4d61469349e5 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:10.554690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-888e5909-721d-49c9-8ab4-4d61469349e5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:11:10.554690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:10.554690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:10.554690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:10.554690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:10.554690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:10.554690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:10.554690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:10.554690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:11:10.554690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:10.554690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:10.554690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:10.554690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:10.554690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:10.554690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:10.554690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:10.555679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:10.555679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:10.555679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:10.555679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:10.555679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:10.555679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:10.555679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:10.555679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:10.555679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:10.555679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:10.555679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:10.555679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:10.555679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:10.555679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:10.555679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:10.555679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:10.556960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:10.556960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:10.556960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:10.556960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:10.556960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:10.556960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:10.556960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:10.556960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:10.556960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:10.556960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:10.556960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:10.556960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:10.556960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:10.556960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:11:10.556960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:10.556960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 358/715] 10.222.0.43 () {56 vars in 1129 bytes} [Wed Jul 20 19:11:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:10.564539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 358/716] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:10.566637 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e5909-721d-49c9-8ab4-4d61469349e5 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:10.568027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fea952-8871-4f5b-bf3c-6b77dc893fbc None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:10.783513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:10.812534 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fea952-8871-4f5b-bf3c-6b77dc893fbc None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:10.843501 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e5909-721d-49c9-8ab4-4d61469349e5 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:10.874781 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9d031-a5d1-43a4-82a0-7ffc59a10a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:10.876753 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9d031-a5d1-43a4-82a0-7ffc59a10a2c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.877556 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9d031-a5d1-43a4-82a0-7ffc59a10a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.878359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9d031-a5d1-43a4-82a0-7ffc59a10a2c None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.879992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9d031-a5d1-43a4-82a0-7ffc59a10a2c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:10.880865 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9d031-a5d1-43a4-82a0-7ffc59a10a2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:10.884589 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9d031-a5d1-43a4-82a0-7ffc59a10a2c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:10.894272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 359/717] 10.222.0.43 () {56 vars in 1124 bytes} [Wed Jul 20 19:11:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:10.904759 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9d031-a5d1-43a4-82a0-7ffc59a10a2c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:10.944268 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbce84a-8e40-4159-a5bb-83c8aeabf12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:10.946928 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbce84a-8e40-4159-a5bb-83c8aeabf12f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.946928 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbce84a-8e40-4159-a5bb-83c8aeabf12f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.946928 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbce84a-8e40-4159-a5bb-83c8aeabf12f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.946928 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbce84a-8e40-4159-a5bb-83c8aeabf12f None admin] RBAC: Adding query filter params (role.id=7fe62c5c771d41c28f790fade1df9357, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=1c944eab8a8948749bb8375787216b09) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:10.946928 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbce84a-8e40-4159-a5bb-83c8aeabf12f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:10.948045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbce84a-8e40-4159-a5bb-83c8aeabf12f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:10.957000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:10.965405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 359/718] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:10 2022] GET /identity/v3/role_assignments?user.id=1c944eab8a8948749bb8375787216b09&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=7fe62c5c771d41c28f790fade1df9357 => generated 608 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:11.070870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9d031-a5d1-43a4-82a0-7ffc59a10a2c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:11.122747 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0888b-8a7b-4486-98e8-0378be693e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:11.123438 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0888b-8a7b-4486-98e8-0378be693e9e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:11.123718 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0888b-8a7b-4486-98e8-0378be693e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:11.123992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0888b-8a7b-4486-98e8-0378be693e9e None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:11.124858 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0888b-8a7b-4486-98e8-0378be693e9e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:11.125163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0888b-8a7b-4486-98e8-0378be693e9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:11.126578 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0888b-8a7b-4486-98e8-0378be693e9e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:11.137785 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 360/719] 10.222.0.43 () {56 vars in 1124 bytes} [Wed Jul 20 19:11:10 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:11.527387 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ab3ad-b991-424e-b6de-9b7fee2aacea None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:11.528070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ab3ad-b991-424e-b6de-9b7fee2aacea None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:11.528593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ab3ad-b991-424e-b6de-9b7fee2aacea None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:11.530224 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 360/720] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:11.536586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c57008-e810-4ee4-8dde-ef7ab02129b2 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:11.537218 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c57008-e810-4ee4-8dde-ef7ab02129b2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:11.537699 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c57008-e810-4ee4-8dde-ef7ab02129b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:11.557037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-79c57008-e810-4ee4-8dde-ef7ab02129b2 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:11.558817 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-79c57008-e810-4ee4-8dde-ef7ab02129b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:11.647292 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 361/721] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:11:11.673881 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ab3ad-b991-424e-b6de-9b7fee2aacea None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:11.688625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d655e-668f-4ef4-a6a7-ca787bfea8aa None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:11.688625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d655e-668f-4ef4-a6a7-ca787bfea8aa None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:11.688935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d655e-668f-4ef4-a6a7-ca787bfea8aa None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:11.690581 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 362/722] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:11.696912 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991f460-fd5d-4644-870e-c67f6ecdbf15 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:11.697272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991f460-fd5d-4644-870e-c67f6ecdbf15 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:11.697467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991f460-fd5d-4644-870e-c67f6ecdbf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:11.717633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8991f460-fd5d-4644-870e-c67f6ecdbf15 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:11.719153 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8991f460-fd5d-4644-870e-c67f6ecdbf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:11.736726 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:11.799443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 363/723] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:11:11.825967 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991f460-fd5d-4644-870e-c67f6ecdbf15 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:11.875207 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ab3ad-b991-424e-b6de-9b7fee2aacea None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:11.921448 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ab393-4e5e-4cbb-a0aa-23a3078df1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:11.922621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ab393-4e5e-4cbb-a0aa-23a3078df1ad None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:11.923112 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ab393-4e5e-4cbb-a0aa-23a3078df1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:11.923588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ab393-4e5e-4cbb-a0aa-23a3078df1ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:11.935839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ab393-4e5e-4cbb-a0aa-23a3078df1ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:11.944132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-112ab393-4e5e-4cbb-a0aa-23a3078df1ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:11:11.944132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:11.944132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:11.944132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:11.944132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:11.944132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:11.944132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:11.944132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:11.944132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:11.944132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:11.944132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:11.944132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:11.944132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:11.944132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:11.944132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:11.944132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:11.945105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:11.945105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:11.945105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:11.945105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:11.945105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:11.945105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:11.945105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:11.945105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:11.945105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:11.945105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:11.945105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:11.945105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:11.945105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:11.945105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:11.945105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:11.945105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:11.945975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:11.945975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:11.945975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:11.945975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:11.945975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:11.945975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:11.945975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:11.945975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:11.945975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:11.945975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:11.945975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:11:11.945975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:11.948869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 361/724] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:11:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:11.954231 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ab393-4e5e-4cbb-a0aa-23a3078df1ad None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:11.977548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:12.018336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991f460-fd5d-4644-870e-c67f6ecdbf15 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:12.087201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcdba69-cb12-4de9-a8a9-9efbdea25a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:12.087201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcdba69-cb12-4de9-a8a9-9efbdea25a8b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:12.087201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcdba69-cb12-4de9-a8a9-9efbdea25a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:12.087201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcdba69-cb12-4de9-a8a9-9efbdea25a8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:12.105774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcdba69-cb12-4de9-a8a9-9efbdea25a8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:12.119485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-7dcdba69-cb12-4de9-a8a9-9efbdea25a8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:12.119485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:12.119485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:12.119485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:12.119485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:12.119485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:12.119485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:12.119485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:12.119485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:12.119485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:12.119485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:12.119485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:12.119485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:12.119485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:12.119485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:12.119485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:12.120551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:12.120551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:12.120551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:12.120551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:12.120551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:12.120551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:12.120551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:12.120551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:12.120551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:12.120551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:12.120551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:12.120551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:12.120551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:12.120551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:12.120551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:12.120551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:12.121404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:12.121404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:12.121404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:12.121404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:12.121404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:12.121404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:12.121404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:12.121404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:12.121404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:12.121404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:12.121404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:12.121404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:12.122508 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 364/725] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:12.129297 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcdba69-cb12-4de9-a8a9-9efbdea25a8b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:12.160578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ab393-4e5e-4cbb-a0aa-23a3078df1ad None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:12.161571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:12.237142 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99feae1-9551-4a14-a426-e87cbc6014dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:12.238333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99feae1-9551-4a14-a426-e87cbc6014dc None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:12.238827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99feae1-9551-4a14-a426-e87cbc6014dc None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:12.239865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99feae1-9551-4a14-a426-e87cbc6014dc None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:12.241074 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99feae1-9551-4a14-a426-e87cbc6014dc None admin] RBAC: Adding query filter params (name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:12.241591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99feae1-9551-4a14-a426-e87cbc6014dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:12.243407 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99feae1-9551-4a14-a426-e87cbc6014dc None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:12.264099 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 362/726] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:11:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:12.269093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99feae1-9551-4a14-a426-e87cbc6014dc None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:12.282150 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcdba69-cb12-4de9-a8a9-9efbdea25a8b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:12.322502 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccebeb-cb68-4c46-bb48-9612ad1e4729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:12.323453 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccebeb-cb68-4c46-bb48-9612ad1e4729 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:12.323742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccebeb-cb68-4c46-bb48-9612ad1e4729 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:12.324019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccebeb-cb68-4c46-bb48-9612ad1e4729 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:12.325271 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccebeb-cb68-4c46-bb48-9612ad1e4729 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:12.325607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccebeb-cb68-4c46-bb48-9612ad1e4729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:12.327226 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccebeb-cb68-4c46-bb48-9612ad1e4729 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:12.343983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 365/727] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:12.355293 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccebeb-cb68-4c46-bb48-9612ad1e4729 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:12.404742 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:12.436251 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99feae1-9551-4a14-a426-e87cbc6014dc None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:12.478804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ce593-6cb1-44d9-a789-e0f82c323ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:12.479821 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ce593-6cb1-44d9-a789-e0f82c323ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:12.480206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ce593-6cb1-44d9-a789-e0f82c323ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:12.480619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ce593-6cb1-44d9-a789-e0f82c323ea4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:12.502554 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ce593-6cb1-44d9-a789-e0f82c323ea4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:12.503940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ce593-6cb1-44d9-a789-e0f82c323ea4 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:12.536599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-367ce593-6cb1-44d9-a789-e0f82c323ea4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:12.536599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:12.536599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:12.536599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:12.536599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:12.536599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:12.536599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:12.536599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:12.536599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:12.536599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:12.536599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:12.536599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:12.536599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:12.536599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:12.536599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:12.537636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:12.538543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:12.538543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:12.538543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:12.538543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:12.538774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:12.540172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 363/728] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:12.552574 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ce593-6cb1-44d9-a789-e0f82c323ea4 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:12.594490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccebeb-cb68-4c46-bb48-9612ad1e4729 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:12.637058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdda57-11ae-4f0a-95fe-4e72bcf27ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:12.637867 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdda57-11ae-4f0a-95fe-4e72bcf27ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:12.638154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdda57-11ae-4f0a-95fe-4e72bcf27ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:12.638428 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdda57-11ae-4f0a-95fe-4e72bcf27ac5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:12.664943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bdda57-11ae-4f0a-95fe-4e72bcf27ac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:12.666653 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bdda57-11ae-4f0a-95fe-4e72bcf27ac5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:12.686862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-e5bdda57-11ae-4f0a-95fe-4e72bcf27ac5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:12.686862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:12.686862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:12.686862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:12.686862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:12.686862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:12.686862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:12.686862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:12.686862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:12.686862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:12.686862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:12.686862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:12.686862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:12.686862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:12.686862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:12.687810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:12.689349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:12.689349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:12.689349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:12.689349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:12.689349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 366/729] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:12.691871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa259f-2ee2-458f-963c-39ad61c2c514 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:12.691871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa259f-2ee2-458f-963c-39ad61c2c514 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:12.691871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa259f-2ee2-458f-963c-39ad61c2c514 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:12.691871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 367/730] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:12.694787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa259f-2ee2-458f-963c-39ad61c2c514 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:12.745879 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ce593-6cb1-44d9-a789-e0f82c323ea4 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:12.778799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:12.796242 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3549e8-6864-4b8d-b9c4-9830d77becc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:12.797177 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3549e8-6864-4b8d-b9c4-9830d77becc0 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:12.797602 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3549e8-6864-4b8d-b9c4-9830d77becc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:12.798017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3549e8-6864-4b8d-b9c4-9830d77becc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:12.799441 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3549e8-6864-4b8d-b9c4-9830d77becc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:12.800137 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3549e8-6864-4b8d-b9c4-9830d77becc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:12.801976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3549e8-6864-4b8d-b9c4-9830d77becc0 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:12.813534 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa259f-2ee2-458f-963c-39ad61c2c514 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:12.819811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 364/731] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:12.821420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fa81a-771c-4449-8f4c-9a89309ce17b None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:12.821856 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fa81a-771c-4449-8f4c-9a89309ce17b None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:12.822874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fa81a-771c-4449-8f4c-9a89309ce17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:12.840649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:12.844588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-c41fa81a-771c-4449-8f4c-9a89309ce17b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:12.846029 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c41fa81a-771c-4449-8f4c-9a89309ce17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:12.859622 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d52b7-1284-44de-9cc6-c899cc76723f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:12.860681 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d52b7-1284-44de-9cc6-c899cc76723f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:12.861571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d52b7-1284-44de-9cc6-c899cc76723f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:12.861860 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d52b7-1284-44de-9cc6-c899cc76723f None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:12.863004 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d52b7-1284-44de-9cc6-c899cc76723f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:12.863330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d52b7-1284-44de-9cc6-c899cc76723f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:12.865040 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d52b7-1284-44de-9cc6-c899cc76723f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:12.881589 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 368/732] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:12.882579 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d52b7-1284-44de-9cc6-c899cc76723f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:12.926414 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 365/733] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:11:12.927698 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41fa81a-771c-4449-8f4c-9a89309ce17b None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:13.030825 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d52b7-1284-44de-9cc6-c899cc76723f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:13.054543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41fa81a-771c-4449-8f4c-9a89309ce17b None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:13.069945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:13.070381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef2f66-a156-4939-869a-4c37e223c237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:13.071610 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef2f66-a156-4939-869a-4c37e223c237 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.071893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef2f66-a156-4939-869a-4c37e223c237 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.072161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef2f66-a156-4939-869a-4c37e223c237 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.081114 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef2f66-a156-4939-869a-4c37e223c237 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:13.082559 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef2f66-a156-4939-869a-4c37e223c237 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:13.096121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dd981-827d-45d0-8e48-c23e85514858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:13.097327 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dd981-827d-45d0-8e48-c23e85514858 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.097814 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dd981-827d-45d0-8e48-c23e85514858 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.098284 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dd981-827d-45d0-8e48-c23e85514858 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.099198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-24ef2f66-a156-4939-869a-4c37e223c237 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:11:13.099198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:13.099198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:13.099198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:13.099198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:13.099198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:13.099198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:13.099198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:13.099198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:13.099198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:13.099198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:13.099198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:13.099198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:13.099198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:13.099198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:13.099198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:13.100094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:13.101065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:13.101065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:13.101065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:13.101065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:13.101065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:13.101065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:13.101065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:13.101065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:13.101065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:11:13.101065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:13.101065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 369/734] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jul 20 19:11:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:13.101714 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef2f66-a156-4939-869a-4c37e223c237 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:13.115270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dd981-827d-45d0-8e48-c23e85514858 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:13.117178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dd981-827d-45d0-8e48-c23e85514858 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:13.133453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-804dd981-827d-45d0-8e48-c23e85514858 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:13.133453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:13.133453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:13.133453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:13.133453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:13.133453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:13.133453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:13.133453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:13.133453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:13.133453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:13.133453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:13.133453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:13.133453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:13.133453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:13.133453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:13.133453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:13.134416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:13.135358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:13.135358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:13.135358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:13.135358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:13.135358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:13.135358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:13.135358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:13.135358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:13.135358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:13.135358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:13.138054 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 366/735] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jul 20 19:11:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:13.139281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dd981-827d-45d0-8e48-c23e85514858 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:13.151511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:13.217158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef2f66-a156-4939-869a-4c37e223c237 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:13.246445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dd981-827d-45d0-8e48-c23e85514858 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:13.257681 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff247b-011d-4055-bb35-9668cf7f6a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:13.258670 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff247b-011d-4055-bb35-9668cf7f6a31 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.258784 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff247b-011d-4055-bb35-9668cf7f6a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.258995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff247b-011d-4055-bb35-9668cf7f6a31 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.259718 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff247b-011d-4055-bb35-9668cf7f6a31 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:13.261258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff247b-011d-4055-bb35-9668cf7f6a31 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:13.284479 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 370/736] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jul 20 19:11:13 2022] POST /identity/v3/services => generated 240 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:13.285852 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff247b-011d-4055-bb35-9668cf7f6a31 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:13.319568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26bbb3-c417-4669-9df9-b97ff7a12659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:13.320893 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26bbb3-c417-4669-9df9-b97ff7a12659 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.322491 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26bbb3-c417-4669-9df9-b97ff7a12659 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.323003 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26bbb3-c417-4669-9df9-b97ff7a12659 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.324801 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26bbb3-c417-4669-9df9-b97ff7a12659 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:13.326274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26bbb3-c417-4669-9df9-b97ff7a12659 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:13.328840 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26bbb3-c417-4669-9df9-b97ff7a12659 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:13.367546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 367/737] 10.222.0.43 () {56 vars in 1149 bytes} [Wed Jul 20 19:11:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:13.374916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26bbb3-c417-4669-9df9-b97ff7a12659 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:13.443561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:13.461879 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff247b-011d-4055-bb35-9668cf7f6a31 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:13.463258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:13.499537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d378d-daae-4c9e-b26e-296697c8d8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:13.500360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d378d-daae-4c9e-b26e-296697c8d8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.500716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d378d-daae-4c9e-b26e-296697c8d8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.501061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d378d-daae-4c9e-b26e-296697c8d8d3 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.502005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d378d-daae-4c9e-b26e-296697c8d8d3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:13.502375 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d378d-daae-4c9e-b26e-296697c8d8d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:13.503906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d378d-daae-4c9e-b26e-296697c8d8d3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:13.526913 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 371/738] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jul 20 19:11:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:13.528306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56b05a-4810-40f2-9d4e-8dd4d0615183 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.528601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56b05a-4810-40f2-9d4e-8dd4d0615183 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.528877 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56b05a-4810-40f2-9d4e-8dd4d0615183 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.530334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 372/739] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:13.535495 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56b05a-4810-40f2-9d4e-8dd4d0615183 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:13.535965 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26bbb3-c417-4669-9df9-b97ff7a12659 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:13.574106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7004481-5fb7-448a-bb5c-41b18de95b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:13.575168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7004481-5fb7-448a-bb5c-41b18de95b11 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.575631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7004481-5fb7-448a-bb5c-41b18de95b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.576083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7004481-5fb7-448a-bb5c-41b18de95b11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.652064 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7004481-5fb7-448a-bb5c-41b18de95b11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:13.654074 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7004481-5fb7-448a-bb5c-41b18de95b11 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:13.673737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56b05a-4810-40f2-9d4e-8dd4d0615183 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:13.684586 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:13.728587 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2533d83-5b6d-479c-a56f-b5f36e7f4758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:13.729785 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2533d83-5b6d-479c-a56f-b5f36e7f4758 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.730075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2533d83-5b6d-479c-a56f-b5f36e7f4758 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.730354 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2533d83-5b6d-479c-a56f-b5f36e7f4758 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.744211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2533d83-5b6d-479c-a56f-b5f36e7f4758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:13.745691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2533d83-5b6d-479c-a56f-b5f36e7f4758 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:13.771311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-c2533d83-5b6d-479c-a56f-b5f36e7f4758 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:13.771311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:13.771311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:13.771311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:13.771311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:13.771311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:13.771311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:13.771311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:13.771311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:13.771311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:13.771311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:13.771311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:13.771311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:13.771311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:13.771311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:13.772278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:13.773501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:13.773501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:13.773501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:13.773501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:13.773501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 373/740] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:13.774288 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3ff5b-c87e-4fb9-836a-69081540bfb4 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.774574 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3ff5b-c87e-4fb9-836a-69081540bfb4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.774851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3ff5b-c87e-4fb9-836a-69081540bfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.780562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 368/741] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:13 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:11:13.782062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7004481-5fb7-448a-bb5c-41b18de95b11 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:13.794732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-01d3ff5b-c87e-4fb9-836a-69081540bfb4 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:13.796575 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-01d3ff5b-c87e-4fb9-836a-69081540bfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:13.807406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:13.896378 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 374/742] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:13 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:11:13.898465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3ff5b-c87e-4fb9-836a-69081540bfb4 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:13.900777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7004481-5fb7-448a-bb5c-41b18de95b11 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:13.930111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:13.941328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b614fe4-1fdd-4f38-8b0d-ec66c2cc1eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:13.942004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b614fe4-1fdd-4f38-8b0d-ec66c2cc1eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.942830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b614fe4-1fdd-4f38-8b0d-ec66c2cc1eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.943107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b614fe4-1fdd-4f38-8b0d-ec66c2cc1eb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.943840 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b614fe4-1fdd-4f38-8b0d-ec66c2cc1eb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:13.944611 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b614fe4-1fdd-4f38-8b0d-ec66c2cc1eb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:13.945991 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b614fe4-1fdd-4f38-8b0d-ec66c2cc1eb1 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:13.962819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 369/743] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:13.962819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab4854-0084-4380-937e-a36b3f94c12c None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.962819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab4854-0084-4380-937e-a36b3f94c12c None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.962819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab4854-0084-4380-937e-a36b3f94c12c None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.962819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 370/744] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:13.962819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab4854-0084-4380-937e-a36b3f94c12c None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:14.015668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3ff5b-c87e-4fb9-836a-69081540bfb4 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:14.058663 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e578c99-33e8-4ea4-b510-49bfabfcf255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:14.059458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e578c99-33e8-4ea4-b510-49bfabfcf255 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:14.059741 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e578c99-33e8-4ea4-b510-49bfabfcf255 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:14.060012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e578c99-33e8-4ea4-b510-49bfabfcf255 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:14.071300 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab4854-0084-4380-937e-a36b3f94c12c None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:14.071460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e578c99-33e8-4ea4-b510-49bfabfcf255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:14.072850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e578c99-33e8-4ea4-b510-49bfabfcf255 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:14.074774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:14.092309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-6e578c99-33e8-4ea4-b510-49bfabfcf255 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:14.092309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:14.092309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:14.092309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:14.092309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:14.092309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:14.092309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:14.092309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:14.092309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:14.092309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:14.092309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:14.092309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:14.092309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:14.092309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:14.092309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:14.093231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:14.094102 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:14.094102 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:14.094102 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:14.094102 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:14.094102 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 375/745] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:14.094395 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e578c99-33e8-4ea4-b510-49bfabfcf255 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:14.112016 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50e3dc-0377-4a12-aeb6-4213c08eb72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:14.112823 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50e3dc-0377-4a12-aeb6-4213c08eb72a None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:14.113174 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50e3dc-0377-4a12-aeb6-4213c08eb72a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:14.113514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50e3dc-0377-4a12-aeb6-4213c08eb72a None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:14.114830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50e3dc-0377-4a12-aeb6-4213c08eb72a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:14.116253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50e3dc-0377-4a12-aeb6-4213c08eb72a None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:14.131688 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 371/746] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jul 20 19:11:13 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:14.133206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f2188-2e74-4a36-aee4-30b85f23baa9 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:14.133496 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f2188-2e74-4a36-aee4-30b85f23baa9 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:14.133777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f2188-2e74-4a36-aee4-30b85f23baa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:14.158705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-709f2188-2e74-4a36-aee4-30b85f23baa9 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:14.159939 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-709f2188-2e74-4a36-aee4-30b85f23baa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:14.219562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:14.244217 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e578c99-33e8-4ea4-b510-49bfabfcf255 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:14.268318 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 372/747] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:14.268318 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f2188-2e74-4a36-aee4-30b85f23baa9 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:14.309167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c12bbf-f1d5-4de7-ac97-b504bfcc66e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:14.310348 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c12bbf-f1d5-4de7-ac97-b504bfcc66e5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:14.310779 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c12bbf-f1d5-4de7-ac97-b504bfcc66e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:14.311079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c12bbf-f1d5-4de7-ac97-b504bfcc66e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:14.322667 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c12bbf-f1d5-4de7-ac97-b504bfcc66e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:14.324135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c12bbf-f1d5-4de7-ac97-b504bfcc66e5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:14.343097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-d6c12bbf-f1d5-4de7-ac97-b504bfcc66e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:14.343097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:14.343097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:14.343097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:14.343097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:14.343097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:14.343097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:14.343097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:14.343097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:14.343097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:14.343097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:14.343097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:14.343097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:14.343097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:14.343097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:14.344091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:14.345181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:14.345181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:14.345181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:14.345181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:14.345181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:14.345181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:14.345181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:14.345181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:14.347450 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 376/748] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:14.348548 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c12bbf-f1d5-4de7-ac97-b504bfcc66e5 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:14.391772 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f2188-2e74-4a36-aee4-30b85f23baa9 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:14.424667 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:14.429030 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f2fb3-bc41-4514-b324-ccc54b534fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:14.429799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f2fb3-bc41-4514-b324-ccc54b534fbb None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:14.430154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f2fb3-bc41-4514-b324-ccc54b534fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:14.430497 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f2fb3-bc41-4514-b324-ccc54b534fbb None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:14.431298 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f2fb3-bc41-4514-b324-ccc54b534fbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:14.431674 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f2fb3-bc41-4514-b324-ccc54b534fbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:14.433392 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f2fb3-bc41-4514-b324-ccc54b534fbb None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:14.452405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 373/749] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:14.453336 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f2fb3-bc41-4514-b324-ccc54b534fbb None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:14.487444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:14.488705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c12bbf-f1d5-4de7-ac97-b504bfcc66e5 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:14.536993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72332e32-9e21-481e-b6b5-f929dddd5e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:14.537796 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72332e32-9e21-481e-b6b5-f929dddd5e9a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:14.538079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72332e32-9e21-481e-b6b5-f929dddd5e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:14.538356 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72332e32-9e21-481e-b6b5-f929dddd5e9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:14.549511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72332e32-9e21-481e-b6b5-f929dddd5e9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:14.558098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-72332e32-9e21-481e-b6b5-f929dddd5e9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:14.558098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:14.558098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:14.558098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:14.558098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:14.558098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:14.558098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:14.558098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:14.558098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:14.558098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:14.558098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:14.558098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:14.558098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:14.558098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:14.558098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:14.558098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:14.559143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:14.559143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:14.559143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:14.559143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:14.559143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:14.559143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:14.559143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:14.559143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:14.559143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:14.559143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:14.559143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:14.559143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:14.559143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:14.559143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:14.559143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:14.559143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:14.560103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:14.560103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:14.560103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:14.560103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:14.560103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:14.560103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:14.560103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:14.560103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:14.560103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:14.560103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:14.560103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:14.560103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:14.564387 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 377/750] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:14.565117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72332e32-9e21-481e-b6b5-f929dddd5e9a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:14.576235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f2fb3-bc41-4514-b324-ccc54b534fbb None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:14.622199 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea6e6c-c64c-4ecd-be5b-918274b74431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:14.623343 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea6e6c-c64c-4ecd-be5b-918274b74431 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:14.623804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea6e6c-c64c-4ecd-be5b-918274b74431 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:14.624465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea6e6c-c64c-4ecd-be5b-918274b74431 None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:14.625929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea6e6c-c64c-4ecd-be5b-918274b74431 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:14.626513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea6e6c-c64c-4ecd-be5b-918274b74431 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:14.628728 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea6e6c-c64c-4ecd-be5b-918274b74431 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:14.644725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 374/751] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:14.645816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea6e6c-c64c-4ecd-be5b-918274b74431 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:14.684794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72332e32-9e21-481e-b6b5-f929dddd5e9a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:14.710038 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:14.726859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1178690-627b-4012-8228-8284f0558102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:14.727654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1178690-627b-4012-8228-8284f0558102 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:14.727935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1178690-627b-4012-8228-8284f0558102 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:14.728205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1178690-627b-4012-8228-8284f0558102 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:14.736903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1178690-627b-4012-8228-8284f0558102 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:14.738340 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1178690-627b-4012-8228-8284f0558102 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:14.751213 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea6e6c-c64c-4ecd-be5b-918274b74431 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:14.757584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-c1178690-627b-4012-8228-8284f0558102 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:14.757584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:14.757584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:14.757584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:14.757584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:14.757584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:14.757584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:14.757584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:14.757584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:14.757584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:14.757584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:14.757584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:14.757584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:14.757584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:14.757584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:14.758442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:14.759341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:14.759341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:14.759341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:14.759341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:14.759341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:14.759341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:14.759341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:14.759341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:14.763426 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 378/752] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:14.764219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1178690-627b-4012-8228-8284f0558102 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:14.764662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:14.798426 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa990d-4361-45d3-a4fc-3ccc47fe36b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:14.799885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa990d-4361-45d3-a4fc-3ccc47fe36b1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:14.800503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa990d-4361-45d3-a4fc-3ccc47fe36b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:14.801111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa990d-4361-45d3-a4fc-3ccc47fe36b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:14.802727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa990d-4361-45d3-a4fc-3ccc47fe36b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:14.803303 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa990d-4361-45d3-a4fc-3ccc47fe36b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:14.805036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa990d-4361-45d3-a4fc-3ccc47fe36b1 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:14.819322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 375/753] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:14.820600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa990d-4361-45d3-a4fc-3ccc47fe36b1 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:14.892324 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1178690-627b-4012-8228-8284f0558102 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:14.929732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa990d-4361-45d3-a4fc-3ccc47fe36b1 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:14.941458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9029677-4fc9-4b12-b2e3-ee2abe82b201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:14.942896 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9029677-4fc9-4b12-b2e3-ee2abe82b201 None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:14.943180 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9029677-4fc9-4b12-b2e3-ee2abe82b201 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:14.943482 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9029677-4fc9-4b12-b2e3-ee2abe82b201 None admin] PATH_INFO: `/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/267b178236e046e4bc694b2da6f86eb0/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:14.947280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9029677-4fc9-4b12-b2e3-ee2abe82b201 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e2a50ebbb48417c911df499d98a0157, user_id=267b178236e046e4bc694b2da6f86eb0, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:14.949806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9029677-4fc9-4b12-b2e3-ee2abe82b201 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:14.968482 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 379/754] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:11:14 2022] PUT /identity/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/267b178236e046e4bc694b2da6f86eb0/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:11:14.969418 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9029677-4fc9-4b12-b2e3-ee2abe82b201 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:14.971718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbee35-3d10-49e6-bb50-0f16b4abaeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:14.972768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbee35-3d10-49e6-bb50-0f16b4abaeca None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:14.973295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbee35-3d10-49e6-bb50-0f16b4abaeca None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:14.973868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbee35-3d10-49e6-bb50-0f16b4abaeca None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:14.974917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bbee35-3d10-49e6-bb50-0f16b4abaeca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:14.975510 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bbee35-3d10-49e6-bb50-0f16b4abaeca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:14.977301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bbee35-3d10-49e6-bb50-0f16b4abaeca None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:14.991807 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 376/755] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:15.001824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbee35-3d10-49e6-bb50-0f16b4abaeca None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:15.023570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:15.025984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:15.139655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbee35-3d10-49e6-bb50-0f16b4abaeca None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:15.168200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9029677-4fc9-4b12-b2e3-ee2abe82b201 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:15.211385 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d61fef-f1ad-40b0-9b5b-0b8ec4f3a068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:15.211862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbd9e2-9509-4faf-97de-53fb1d4bb5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:15.212054 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d61fef-f1ad-40b0-9b5b-0b8ec4f3a068 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:15.212356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d61fef-f1ad-40b0-9b5b-0b8ec4f3a068 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:15.212637 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d61fef-f1ad-40b0-9b5b-0b8ec4f3a068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:15.213457 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbd9e2-9509-4faf-97de-53fb1d4bb5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:15.213577 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d61fef-f1ad-40b0-9b5b-0b8ec4f3a068 None admin] RBAC: Adding query filter params (role.id=7fe62c5c771d41c28f790fade1df9357, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=2d514236db644010859fc18250f5894e) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:15.213783 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d61fef-f1ad-40b0-9b5b-0b8ec4f3a068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:15.214239 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbd9e2-9509-4faf-97de-53fb1d4bb5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:15.214701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbd9e2-9509-4faf-97de-53fb1d4bb5d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:15.215209 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d61fef-f1ad-40b0-9b5b-0b8ec4f3a068 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:15.222986 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 377/756] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:14 2022] GET /identity/v3/role_assignments?user.id=2d514236db644010859fc18250f5894e&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=7fe62c5c771d41c28f790fade1df9357 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:15.240230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbd9e2-9509-4faf-97de-53fb1d4bb5d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:15.242212 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbd9e2-9509-4faf-97de-53fb1d4bb5d4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:15.262519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-65cbd9e2-9509-4faf-97de-53fb1d4bb5d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:15.262519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:15.262519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:15.262519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:15.262519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:15.262519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:15.262519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:15.262519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:15.262519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:15.262519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:15.262519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:15.262519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:15.262519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:15.262519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:15.262519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:15.263472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:15.264352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:15.264352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:15.264352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:15.264352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:15.266689 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 380/757] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:15.274622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d61fef-f1ad-40b0-9b5b-0b8ec4f3a068 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:15.386655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:15.418039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d61fef-f1ad-40b0-9b5b-0b8ec4f3a068 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:15.455754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116f599-0be4-44aa-8f94-a09cc5dfa40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:15.456810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116f599-0be4-44aa-8f94-a09cc5dfa40a None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:15.457327 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116f599-0be4-44aa-8f94-a09cc5dfa40a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:15.458312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116f599-0be4-44aa-8f94-a09cc5dfa40a None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:15.459250 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116f599-0be4-44aa-8f94-a09cc5dfa40a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:15.459762 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116f599-0be4-44aa-8f94-a09cc5dfa40a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:15.461341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116f599-0be4-44aa-8f94-a09cc5dfa40a None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:15.466713 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f776e3-9797-475c-b8fa-b064048e29a2 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:15.467282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f776e3-9797-475c-b8fa-b064048e29a2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:15.467773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f776e3-9797-475c-b8fa-b064048e29a2 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:15.469374 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 381/758] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:15.475393 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a853a-6b52-4113-b7dc-2aedec5de60b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:15.475689 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a853a-6b52-4113-b7dc-2aedec5de60b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:15.475971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a853a-6b52-4113-b7dc-2aedec5de60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:15.481321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 378/759] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:15.489032 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116f599-0be4-44aa-8f94-a09cc5dfa40a None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:15.498084 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ab1a853a-6b52-4113-b7dc-2aedec5de60b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:15.499288 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ab1a853a-6b52-4113-b7dc-2aedec5de60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:15.630643 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 382/760] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:15.655972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116f599-0be4-44aa-8f94-a09cc5dfa40a None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:15.688330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a853a-6b52-4113-b7dc-2aedec5de60b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:15.716624 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:15.716813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b89468-73db-4963-b7cd-77a399bbc8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:15.717437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b89468-73db-4963-b7cd-77a399bbc8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:15.717723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b89468-73db-4963-b7cd-77a399bbc8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:15.718121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b89468-73db-4963-b7cd-77a399bbc8e1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:15.729925 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b89468-73db-4963-b7cd-77a399bbc8e1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:15.731382 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b89468-73db-4963-b7cd-77a399bbc8e1 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:15.735152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:15.762640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-88b89468-73db-4963-b7cd-77a399bbc8e1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:11:15.762640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:15.762640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:15.762640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:15.762640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:15.762640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:15.762640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:15.762640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:15.762640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:15.762640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:15.762640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:15.762640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:15.762640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:15.762640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:15.762640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:15.762640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:15.763625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:15.764675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:15.764675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:15.764675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:15.764675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:15.764675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:15.764675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:15.764675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:15.764675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:15.764675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:11:15.764675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:15.765199 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 379/761] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:15.771295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b89468-73db-4963-b7cd-77a399bbc8e1 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:15.871041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a853a-6b52-4113-b7dc-2aedec5de60b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:15.941697 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a3323-b129-4348-9696-84e57630edf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:15.942517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a3323-b129-4348-9696-84e57630edf2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:15.942737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a3323-b129-4348-9696-84e57630edf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:15.943010 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a3323-b129-4348-9696-84e57630edf2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:15.943676 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a3323-b129-4348-9696-84e57630edf2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:15.945428 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a3323-b129-4348-9696-84e57630edf2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:15.961336 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b89468-73db-4963-b7cd-77a399bbc8e1 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-ca9a3323-b129-4348-9696-84e57630edf2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:15.963318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:15.964312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:15.964312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:15.964312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:15.964312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:15.964312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:15.964312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:15.964312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:15.964312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:15.964312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:15.964312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:15.964312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:15.964312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:15.964312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:15.964312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:15.964312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:15.964312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:15.965161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:15.965161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:15.965161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:15.965161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:15.965161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:15.965161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:15.965161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:15.965161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:15.965161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:15.965161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:15.965161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:15.965161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:15.965161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:11:15.965161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:15.969701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 383/762] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:15 2022] GET /identity/v3/services/network => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:15.970970 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a3323-b129-4348-9696-84e57630edf2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:16.038944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5ce12-019a-478b-a413-9beda01c17cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:16.039631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5ce12-019a-478b-a413-9beda01c17cf None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:16.039917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5ce12-019a-478b-a413-9beda01c17cf None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:16.040192 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5ce12-019a-478b-a413-9beda01c17cf None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:16.041078 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5ce12-019a-478b-a413-9beda01c17cf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:16.041391 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5ce12-019a-478b-a413-9beda01c17cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:16.042810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5ce12-019a-478b-a413-9beda01c17cf None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:16.079958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 380/763] 10.222.0.43 () {56 vars in 1135 bytes} [Wed Jul 20 19:11:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:16.100660 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5ce12-019a-478b-a413-9beda01c17cf None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:16.106686 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:16.127764 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a3323-b129-4348-9696-84e57630edf2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:16.163108 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16cffd-fcab-41c0-929d-14a868f9d323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:16.164018 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16cffd-fcab-41c0-929d-14a868f9d323 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:16.164318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16cffd-fcab-41c0-929d-14a868f9d323 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:16.164606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16cffd-fcab-41c0-929d-14a868f9d323 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:16.165497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16cffd-fcab-41c0-929d-14a868f9d323 None admin] RBAC: Adding query filter params (name=network) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:16.165811 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16cffd-fcab-41c0-929d-14a868f9d323 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:16.167192 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16cffd-fcab-41c0-929d-14a868f9d323 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:16.176923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 384/764] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:16.178552 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16cffd-fcab-41c0-929d-14a868f9d323 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:16.198631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:16.251866 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5ce12-019a-478b-a413-9beda01c17cf None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:16.310245 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16cffd-fcab-41c0-929d-14a868f9d323 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:16.358769 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a0af3-40ce-4df5-bf16-a95c7c14bc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:16.359738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a0af3-40ce-4df5-bf16-a95c7c14bc1e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:16.360216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a0af3-40ce-4df5-bf16-a95c7c14bc1e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:16.360721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a0af3-40ce-4df5-bf16-a95c7c14bc1e None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:16.361695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a0af3-40ce-4df5-bf16-a95c7c14bc1e None admin] RBAC: Adding query filter params (type=network) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:16.362198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a0af3-40ce-4df5-bf16-a95c7c14bc1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:16.363770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a0af3-40ce-4df5-bf16-a95c7c14bc1e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:16.365064 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5bef9-7394-44df-a292-02acd843e24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:16.365765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5bef9-7394-44df-a292-02acd843e24f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:16.366054 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5bef9-7394-44df-a292-02acd843e24f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:16.366457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5bef9-7394-44df-a292-02acd843e24f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:16.371434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 385/765] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:16.377293 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a0af3-40ce-4df5-bf16-a95c7c14bc1e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:16.397430 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:16.499847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5bef9-7394-44df-a292-02acd843e24f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:16.501493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5bef9-7394-44df-a292-02acd843e24f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:16.531319 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a0af3-40ce-4df5-bf16-a95c7c14bc1e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:16.560698 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:16.567472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cf16c-e856-49ef-bfc1-1ad28e905e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:16.568439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cf16c-e856-49ef-bfc1-1ad28e905e9b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:16.568916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cf16c-e856-49ef-bfc1-1ad28e905e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:16.569380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cf16c-e856-49ef-bfc1-1ad28e905e9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:16.570475 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485cf16c-e856-49ef-bfc1-1ad28e905e9b None admin] RBAC: Adding query filter params (interface=public, service_id=731dba4dcc494f639d1567073a0f694a, region_id=RegionOne) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:16.570984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485cf16c-e856-49ef-bfc1-1ad28e905e9b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:16.572574 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485cf16c-e856-49ef-bfc1-1ad28e905e9b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:16.581006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 386/766] 10.222.0.43 () {56 vars in 1251 bytes} [Wed Jul 20 19:11:16 2022] GET /identity/v3/endpoints?service_id=731dba4dcc494f639d1567073a0f694a&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:16.582717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73233106-4190-4e94-a92d-70cf821441c5 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:16.583287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73233106-4190-4e94-a92d-70cf821441c5 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:16.583778 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73233106-4190-4e94-a92d-70cf821441c5 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:16.585555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 387/767] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:16.586442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73233106-4190-4e94-a92d-70cf821441c5 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:16.613546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 381/768] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:16 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 19:11:16.615692 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff598f4d-2d5d-4418-bb34-fa2db8cf6024 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:16.615990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff598f4d-2d5d-4418-bb34-fa2db8cf6024 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:16.616270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff598f4d-2d5d-4418-bb34-fa2db8cf6024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:16.634703 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-ff598f4d-2d5d-4418-bb34-fa2db8cf6024 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:16.636608 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ff598f4d-2d5d-4418-bb34-fa2db8cf6024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:16.668618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:16.702384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73233106-4190-4e94-a92d-70cf821441c5 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:16.724635 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 382/769] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:16.725562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff598f4d-2d5d-4418-bb34-fa2db8cf6024 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:16.741835 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc131b97-3f09-4267-b4d2-3d7aa37ef1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:16.742871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc131b97-3f09-4267-b4d2-3d7aa37ef1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:16.743352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc131b97-3f09-4267-b4d2-3d7aa37ef1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:16.743819 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc131b97-3f09-4267-b4d2-3d7aa37ef1f6 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:16.744810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc131b97-3f09-4267-b4d2-3d7aa37ef1f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:16.746619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc131b97-3f09-4267-b4d2-3d7aa37ef1f6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:16.753252 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 388/770] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jul 20 19:11:16 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:16.762324 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc131b97-3f09-4267-b4d2-3d7aa37ef1f6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:16.843746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:16.862386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff598f4d-2d5d-4418-bb34-fa2db8cf6024 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:16.901438 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe1a58-c6b1-45d6-be64-137240112cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:16.902262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe1a58-c6b1-45d6-be64-137240112cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:16.902581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe1a58-c6b1-45d6-be64-137240112cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:16.903413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe1a58-c6b1-45d6-be64-137240112cc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:16.916563 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe1a58-c6b1-45d6-be64-137240112cc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:16.918045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe1a58-c6b1-45d6-be64-137240112cc2 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:16.937987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-dcfe1a58-c6b1-45d6-be64-137240112cc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:16.937987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:16.937987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:16.937987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:16.937987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:16.937987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:16.937987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:16.937987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:16.937987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:16.937987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:16.937987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:16.937987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:16.937987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:16.937987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:16.937987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:16.938969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:16.939840 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:16.939840 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:16.939840 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:16.939840 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:16.939840 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 383/771] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:16.945374 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe1a58-c6b1-45d6-be64-137240112cc2 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:16.961566 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc131b97-3f09-4267-b4d2-3d7aa37ef1f6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:17.003061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb779d8-fd29-4085-9fc1-80fd674d8110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:17.004417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb779d8-fd29-4085-9fc1-80fd674d8110 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.005094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb779d8-fd29-4085-9fc1-80fd674d8110 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.008089 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb779d8-fd29-4085-9fc1-80fd674d8110 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.009094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb779d8-fd29-4085-9fc1-80fd674d8110 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:17.010792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb779d8-fd29-4085-9fc1-80fd674d8110 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:17.014467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:17.028307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-3fb779d8-fd29-4085-9fc1-80fd674d8110 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:11:17.028307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:17.028307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:17.028307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:17.028307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:17.028307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:17.028307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:17.028307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:17.028307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:11:17.028307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:17.028307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:17.028307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:17.028307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:17.028307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:17.028307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:17.028307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:17.029328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:17.029328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:17.029328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:17.029328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:17.029328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:17.029328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:17.029328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:17.029328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:17.029328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:17.029328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:17.029328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:17.029328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:17.029328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:17.029328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:17.029328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:17.029328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:17.030140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:17.030140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:17.030140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:17.030140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:17.030140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:17.030140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:17.030140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:17.030140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:17.030140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:17.030140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:17.030140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:17.030140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:17.030140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:17.030140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:11:17.030140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:17.030140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 389/772] 10.222.0.43 () {56 vars in 1129 bytes} [Wed Jul 20 19:11:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:17.037954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb779d8-fd29-4085-9fc1-80fd674d8110 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:17.066062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe1a58-c6b1-45d6-be64-137240112cc2 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:17.105723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4fc94-f427-43f9-9012-0eba6a279345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:17.106708 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4fc94-f427-43f9-9012-0eba6a279345 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.107204 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4fc94-f427-43f9-9012-0eba6a279345 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.107729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4fc94-f427-43f9-9012-0eba6a279345 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.108810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4fc94-f427-43f9-9012-0eba6a279345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:17.109304 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4fc94-f427-43f9-9012-0eba6a279345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:17.110929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4fc94-f427-43f9-9012-0eba6a279345 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:17.124701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:17.127272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 384/773] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:17.131420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4fc94-f427-43f9-9012-0eba6a279345 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:17.162582 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb779d8-fd29-4085-9fc1-80fd674d8110 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:17.199537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f09b41-e1a2-4c40-883a-1789787b0ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:17.200325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f09b41-e1a2-4c40-883a-1789787b0ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.200667 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f09b41-e1a2-4c40-883a-1789787b0ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.201003 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f09b41-e1a2-4c40-883a-1789787b0ef4 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.201808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f09b41-e1a2-4c40-883a-1789787b0ef4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:17.202178 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f09b41-e1a2-4c40-883a-1789787b0ef4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:17.203764 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f09b41-e1a2-4c40-883a-1789787b0ef4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:17.210251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 390/774] 10.222.0.43 () {56 vars in 1124 bytes} [Wed Jul 20 19:11:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:17.217619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f09b41-e1a2-4c40-883a-1789787b0ef4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:17.256896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4fc94-f427-43f9-9012-0eba6a279345 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:17.294945 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:17.322792 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6526b8-7ad6-44b7-abdf-a0e40b4b158b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:17.323599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6526b8-7ad6-44b7-abdf-a0e40b4b158b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.323984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6526b8-7ad6-44b7-abdf-a0e40b4b158b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.324268 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6526b8-7ad6-44b7-abdf-a0e40b4b158b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.334864 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6526b8-7ad6-44b7-abdf-a0e40b4b158b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:17.340926 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6526b8-7ad6-44b7-abdf-a0e40b4b158b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:17.343656 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f09b41-e1a2-4c40-883a-1789787b0ef4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:17.378096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-de6526b8-7ad6-44b7-abdf-a0e40b4b158b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:17.378096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:17.378096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:17.378096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:17.378096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:17.378096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:17.378096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:17.378096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:17.378096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:17.378096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:17.378096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:17.378096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:17.378096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:17.378096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:17.378096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:17.379055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:17.381465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:17.381465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:17.381465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:17.381465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:17.381465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:17.381465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:17.381465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:17.381465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:17.381465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 385/775] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:17.389765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6526b8-7ad6-44b7-abdf-a0e40b4b158b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:17.399358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cb0a6-f881-4d38-be1e-275fe43d7a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:17.400181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cb0a6-f881-4d38-be1e-275fe43d7a06 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.400502 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cb0a6-f881-4d38-be1e-275fe43d7a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.400776 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cb0a6-f881-4d38-be1e-275fe43d7a06 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.401535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133cb0a6-f881-4d38-be1e-275fe43d7a06 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:17.401842 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133cb0a6-f881-4d38-be1e-275fe43d7a06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:17.403206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133cb0a6-f881-4d38-be1e-275fe43d7a06 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:17.408908 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 391/776] 10.222.0.43 () {56 vars in 1124 bytes} [Wed Jul 20 19:11:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:17.415638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cb0a6-f881-4d38-be1e-275fe43d7a06 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:17.476883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:17.521085 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6526b8-7ad6-44b7-abdf-a0e40b4b158b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:17.535781 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cb0a6-f881-4d38-be1e-275fe43d7a06 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:17.557704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44b8a2-b9e1-484d-add4-f99f10349042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:17.558627 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44b8a2-b9e1-484d-add4-f99f10349042 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.558909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44b8a2-b9e1-484d-add4-f99f10349042 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.559179 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44b8a2-b9e1-484d-add4-f99f10349042 None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.560087 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44b8a2-b9e1-484d-add4-f99f10349042 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:17.560545 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44b8a2-b9e1-484d-add4-f99f10349042 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:17.561981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44b8a2-b9e1-484d-add4-f99f10349042 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:17.563577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:17.572658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3294daf-2fb3-475d-bef4-3d7467611715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:17.573410 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3294daf-2fb3-475d-bef4-3d7467611715 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.573769 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3294daf-2fb3-475d-bef4-3d7467611715 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.574122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3294daf-2fb3-475d-bef4-3d7467611715 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.574993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3294daf-2fb3-475d-bef4-3d7467611715 None admin] RBAC: Adding query filter params (interface=public, service_id=15396f94e16041e9a5455175e771cb78, region_id=RegionOne) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:17.575384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3294daf-2fb3-475d-bef4-3d7467611715 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:17.576855 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3294daf-2fb3-475d-bef4-3d7467611715 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:17.582165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 386/777] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:17.583666 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 392/778] 10.222.0.43 () {56 vars in 1251 bytes} [Wed Jul 20 19:11:17 2022] GET /identity/v3/endpoints?service_id=15396f94e16041e9a5455175e771cb78&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:17.584163 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112cd3e-3129-4d14-a8e7-454c5a0ea0d0 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.584483 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112cd3e-3129-4d14-a8e7-454c5a0ea0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.584714 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3294daf-2fb3-475d-bef4-3d7467611715 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:17.584832 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112cd3e-3129-4d14-a8e7-454c5a0ea0d0 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.586364 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 387/779] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:17.592841 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c5dd0-ce89-45c0-b3c3-1ab0a1dcb7bb None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.593203 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c5dd0-ce89-45c0-b3c3-1ab0a1dcb7bb None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.593550 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c5dd0-ce89-45c0-b3c3-1ab0a1dcb7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.614715 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-646c5dd0-ce89-45c0-b3c3-1ab0a1dcb7bb None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:17.616307 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-646c5dd0-ce89-45c0-b3c3-1ab0a1dcb7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:17.698957 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3294daf-2fb3-475d-bef4-3d7467611715 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:17.708426 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 388/780] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:17.709790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c5dd0-ce89-45c0-b3c3-1ab0a1dcb7bb None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:17.739515 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f715ee9-3d7b-405c-8f5c-33061fc54688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:17.740175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f715ee9-3d7b-405c-8f5c-33061fc54688 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.740490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f715ee9-3d7b-405c-8f5c-33061fc54688 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.740760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f715ee9-3d7b-405c-8f5c-33061fc54688 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.741668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f715ee9-3d7b-405c-8f5c-33061fc54688 None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=1c944eab8a8948749bb8375787216b09) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:17.741982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f715ee9-3d7b-405c-8f5c-33061fc54688 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:17.743637 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f715ee9-3d7b-405c-8f5c-33061fc54688 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:17.750996 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 393/781] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:17 2022] GET /identity/v3/role_assignments?user.id=1c944eab8a8948749bb8375787216b09&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:17.758245 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027fa89-8ef7-48be-9060-f26a78082ae5 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.759045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027fa89-8ef7-48be-9060-f26a78082ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.760096 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027fa89-8ef7-48be-9060-f26a78082ae5 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.762486 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 394/782] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:17.763711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027fa89-8ef7-48be-9060-f26a78082ae5 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:17.766653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:17.836728 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c5dd0-ce89-45c0-b3c3-1ab0a1dcb7bb None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:17.858827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:17.878310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65eafc3-8a74-4205-bcef-c2bf650a532a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:17.878992 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65eafc3-8a74-4205-bcef-c2bf650a532a None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.879277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65eafc3-8a74-4205-bcef-c2bf650a532a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.880088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65eafc3-8a74-4205-bcef-c2bf650a532a None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.881135 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65eafc3-8a74-4205-bcef-c2bf650a532a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:17.883543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65eafc3-8a74-4205-bcef-c2bf650a532a None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:17.896529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 389/783] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jul 20 19:11:17 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:17.898421 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be588bb-0fa0-46e9-a4fe-969a3a8e7d14 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.900667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be588bb-0fa0-46e9-a4fe-969a3a8e7d14 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.901945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be588bb-0fa0-46e9-a4fe-969a3a8e7d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.933910 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-1be588bb-0fa0-46e9-a4fe-969a3a8e7d14 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:17.935252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-1be588bb-0fa0-46e9-a4fe-969a3a8e7d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:17.935486 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027fa89-8ef7-48be-9060-f26a78082ae5 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:18.003700 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466204d-48d4-4299-9f43-47c30a6de21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:18.006381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466204d-48d4-4299-9f43-47c30a6de21b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:18.007245 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466204d-48d4-4299-9f43-47c30a6de21b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:18.008346 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466204d-48d4-4299-9f43-47c30a6de21b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:18.033793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c466204d-48d4-4299-9f43-47c30a6de21b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:18.052815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-c466204d-48d4-4299-9f43-47c30a6de21b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:18.052815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:18.052815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:18.052815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:18.052815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:18.052815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:18.052815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:18.052815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:18.052815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:18.052815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:18.052815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:18.052815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:18.052815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:18.052815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:18.052815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:18.052815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:18.053857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:18.053857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:18.053857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:18.053857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:18.053857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:18.053857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:18.053857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:18.053857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:18.053857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:18.053857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:18.053857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:18.053857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:18.053857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:18.053857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:18.053857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:18.053857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:18.054717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:18.054717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:18.054717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:18.054717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:18.054717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:18.054717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:18.054717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:18.054717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:18.054717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:18.054717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:18.054717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:18.054717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:18.058122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 395/784] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:18.065133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466204d-48d4-4299-9f43-47c30a6de21b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:18.076776 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 390/785] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:18.101825 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be588bb-0fa0-46e9-a4fe-969a3a8e7d14 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:18.158455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:18.189679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466204d-48d4-4299-9f43-47c30a6de21b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:18.212504 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:18.237837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be588bb-0fa0-46e9-a4fe-969a3a8e7d14 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:18.250551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5b9b7-0e89-49b9-8abc-afc56c8e8131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:18.251230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5b9b7-0e89-49b9-8abc-afc56c8e8131 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:18.251511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5b9b7-0e89-49b9-8abc-afc56c8e8131 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:18.251782 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5b9b7-0e89-49b9-8abc-afc56c8e8131 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:18.252569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5b9b7-0e89-49b9-8abc-afc56c8e8131 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:18.252876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5b9b7-0e89-49b9-8abc-afc56c8e8131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:18.254263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5b9b7-0e89-49b9-8abc-afc56c8e8131 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:18.269323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 396/786] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:18.281380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5b9b7-0e89-49b9-8abc-afc56c8e8131 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:18.297363 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f51cb-eae1-4088-969e-3ea1db22c82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:18.299467 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f51cb-eae1-4088-969e-3ea1db22c82d None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:18.300002 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f51cb-eae1-4088-969e-3ea1db22c82d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:18.300543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f51cb-eae1-4088-969e-3ea1db22c82d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:18.314330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f51cb-eae1-4088-969e-3ea1db22c82d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:18.327749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-2b3f51cb-eae1-4088-969e-3ea1db22c82d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:11:18.327749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:18.327749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:18.327749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:18.327749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:18.327749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:18.327749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:18.327749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:18.327749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:18.327749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:18.327749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:18.327749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:18.327749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:18.327749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:18.327749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:18.327749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:18.328780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:18.328780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:18.328780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:18.328780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:18.328780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:18.328780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:18.328780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:18.328780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:18.328780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:18.328780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:18.328780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:18.328780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:18.328780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:18.328780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:18.328780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:18.328780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:18.329622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:18.329622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:18.329622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:18.329622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:18.329622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:18.329622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:18.329622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:18.329622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:18.329622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:18.329622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:18.329622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:11:18.329622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:18.336684 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 391/787] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:11:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:18.339430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f51cb-eae1-4088-969e-3ea1db22c82d None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:18.467545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5b9b7-0e89-49b9-8abc-afc56c8e8131 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:18.505503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f874172-280c-49bc-b0c6-a973af8174c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:18.506351 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f874172-280c-49bc-b0c6-a973af8174c4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:18.506639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f874172-280c-49bc-b0c6-a973af8174c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:18.506914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f874172-280c-49bc-b0c6-a973af8174c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:18.518232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f874172-280c-49bc-b0c6-a973af8174c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:18.519704 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f874172-280c-49bc-b0c6-a973af8174c4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:18.537519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-8f874172-280c-49bc-b0c6-a973af8174c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:18.537519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:18.537519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:18.537519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:18.537519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:18.537519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:18.537519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:18.537519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:18.537519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:18.537519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:18.537519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:18.537519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:18.537519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:18.537519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:18.537519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:18.538465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:18.539532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:18.539532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:18.539532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:18.539532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:18.544128 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 397/788] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:18.545927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f874172-280c-49bc-b0c6-a973af8174c4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:18.561147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f51cb-eae1-4088-969e-3ea1db22c82d None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:18.563307 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:18.592649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:18.631668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51de00a-a991-4669-b5b4-3214983e0ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:18.632600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51de00a-a991-4669-b5b4-3214983e0ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:18.632887 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51de00a-a991-4669-b5b4-3214983e0ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:18.633164 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51de00a-a991-4669-b5b4-3214983e0ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:18.633924 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51de00a-a991-4669-b5b4-3214983e0ba0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:18.634253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51de00a-a991-4669-b5b4-3214983e0ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:18.635672 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51de00a-a991-4669-b5b4-3214983e0ba0 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:18.656094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 392/789] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:11:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:18.677268 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51de00a-a991-4669-b5b4-3214983e0ba0 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:18.712015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f874172-280c-49bc-b0c6-a973af8174c4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:18.747810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ea6b9-fca2-4dfc-883c-968f563fee65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:18.748447 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ea6b9-fca2-4dfc-883c-968f563fee65 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:18.748732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ea6b9-fca2-4dfc-883c-968f563fee65 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:18.749142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ea6b9-fca2-4dfc-883c-968f563fee65 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:18.749874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ea6b9-fca2-4dfc-883c-968f563fee65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:18.750183 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ea6b9-fca2-4dfc-883c-968f563fee65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:18.751561 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ea6b9-fca2-4dfc-883c-968f563fee65 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:18.768460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 398/790] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:18.769668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ea6b9-fca2-4dfc-883c-968f563fee65 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:18.871054 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51de00a-a991-4669-b5b4-3214983e0ba0 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:18.875000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ea6b9-fca2-4dfc-883c-968f563fee65 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:18.900701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:18.920087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718aff66-27ea-4d49-b2a9-53dd53c0a3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:18.920913 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718aff66-27ea-4d49-b2a9-53dd53c0a3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:18.921199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718aff66-27ea-4d49-b2a9-53dd53c0a3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:18.921473 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718aff66-27ea-4d49-b2a9-53dd53c0a3f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:18.936646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718aff66-27ea-4d49-b2a9-53dd53c0a3f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:18.938103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718aff66-27ea-4d49-b2a9-53dd53c0a3f8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:18.944049 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffcd4a-49a4-460e-af55-b685289da8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:18.944995 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffcd4a-49a4-460e-af55-b685289da8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:18.945299 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffcd4a-49a4-460e-af55-b685289da8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:18.945497 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffcd4a-49a4-460e-af55-b685289da8c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:18.968247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffcd4a-49a4-460e-af55-b685289da8c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:18.969706 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffcd4a-49a4-460e-af55-b685289da8c0 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:18.973677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-718aff66-27ea-4d49-b2a9-53dd53c0a3f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:18.973677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:18.973677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:18.973677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:18.973677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:18.973677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:18.973677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:18.973677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:18.973677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:18.973677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:18.973677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:18.973677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:18.973677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:18.973677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:18.973677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:18.973677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:18.974981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:18.975920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:18.975920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:18.975920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:18.975920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:18.975920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:18.975920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:18.975920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:18.975920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:18.975920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:18.975920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:18.975920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 399/791] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jul 20 19:11:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:18.981613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718aff66-27ea-4d49-b2a9-53dd53c0a3f8 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:19.008215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-67ffcd4a-49a4-460e-af55-b685289da8c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:19.008215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:19.008215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:19.008215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:19.008215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:19.008215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:19.008215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:19.008215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:19.008215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:19.008215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:19.008215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:19.008215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:19.008215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:19.008215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:19.008215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:19.010357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:19.012518 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:19.012518 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:19.012518 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:19.012518 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:19.018192 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 393/792] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:19.029469 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffcd4a-49a4-460e-af55-b685289da8c0 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:19.050448 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:19.187406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718aff66-27ea-4d49-b2a9-53dd53c0a3f8 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:19.211062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffcd4a-49a4-460e-af55-b685289da8c0 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:19.224219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525eb04-6094-40ab-ab35-f4af4e5f3634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:19.224944 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525eb04-6094-40ab-ab35-f4af4e5f3634 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:19.225227 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525eb04-6094-40ab-ab35-f4af4e5f3634 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:19.225612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525eb04-6094-40ab-ab35-f4af4e5f3634 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:19.226519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525eb04-6094-40ab-ab35-f4af4e5f3634 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:19.226889 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525eb04-6094-40ab-ab35-f4af4e5f3634 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:19.228516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525eb04-6094-40ab-ab35-f4af4e5f3634 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:19.252694 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 400/793] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jul 20 19:11:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:19.254203 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525eb04-6094-40ab-ab35-f4af4e5f3634 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:19.266219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:19.286789 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e545ea-39bf-471e-bf3a-03f3e03dfb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:19.287851 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e545ea-39bf-471e-bf3a-03f3e03dfb57 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:19.288378 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e545ea-39bf-471e-bf3a-03f3e03dfb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:19.288850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e545ea-39bf-471e-bf3a-03f3e03dfb57 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:19.289806 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e545ea-39bf-471e-bf3a-03f3e03dfb57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:19.290310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e545ea-39bf-471e-bf3a-03f3e03dfb57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:19.291873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e545ea-39bf-471e-bf3a-03f3e03dfb57 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:19.322787 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 394/794] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:19.347049 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e545ea-39bf-471e-bf3a-03f3e03dfb57 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:19.370992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525eb04-6094-40ab-ab35-f4af4e5f3634 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:19.408192 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aefcd0-65b2-40aa-a77d-b89976dc07f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:19.408918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aefcd0-65b2-40aa-a77d-b89976dc07f2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:19.409204 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aefcd0-65b2-40aa-a77d-b89976dc07f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:19.409478 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aefcd0-65b2-40aa-a77d-b89976dc07f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:19.484267 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aefcd0-65b2-40aa-a77d-b89976dc07f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:19.485578 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aefcd0-65b2-40aa-a77d-b89976dc07f2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:19.496119 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:19.507042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:19.530853 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e545ea-39bf-471e-bf3a-03f3e03dfb57 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:19.566229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ca5ef-059a-48bc-8695-9fa40d90f727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:19.567758 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ca5ef-059a-48bc-8695-9fa40d90f727 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:19.568172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ca5ef-059a-48bc-8695-9fa40d90f727 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:19.568905 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ca5ef-059a-48bc-8695-9fa40d90f727 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:19.576207 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ca5ef-059a-48bc-8695-9fa40d90f727 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:19.577798 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ca5ef-059a-48bc-8695-9fa40d90f727 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:19.594215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-544ca5ef-059a-48bc-8695-9fa40d90f727 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:11:19.594215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:19.594215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:19.594215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:19.594215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:19.594215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:19.594215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:19.594215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:19.594215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:19.594215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:19.594215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:19.594215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:19.594215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:19.594215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:19.594215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:19.594215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:19.595182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:19.596123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:19.596123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:19.596123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:19.596123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:19.596123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:19.596123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:19.596123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:19.596123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:19.596123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:11:19.596123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:19.603624 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 395/796] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jul 20 19:11:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:19.603624 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbbbdb-5497-41df-9517-7f9fe6b58602 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:19.603624 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbbbdb-5497-41df-9517-7f9fe6b58602 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:19.603624 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbbbdb-5497-41df-9517-7f9fe6b58602 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:19.604095 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 396/797] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:19.604274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 401/797] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:19 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:11:19.605225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbbbdb-5497-41df-9517-7f9fe6b58602 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:19.605914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aefcd0-65b2-40aa-a77d-b89976dc07f2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:19.713401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aefcd0-65b2-40aa-a77d-b89976dc07f2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:19.718504 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbbbdb-5497-41df-9517-7f9fe6b58602 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:19.724589 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:19.751647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303b9f9-9e06-41fb-bc35-b4ec125a841f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:19.752964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303b9f9-9e06-41fb-bc35-b4ec125a841f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:19.753555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303b9f9-9e06-41fb-bc35-b4ec125a841f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:19.754097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303b9f9-9e06-41fb-bc35-b4ec125a841f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:19.756802 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd369ef-9ff1-47e7-a666-791f98d907f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:19.757615 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd369ef-9ff1-47e7-a666-791f98d907f2 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:19.757961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd369ef-9ff1-47e7-a666-791f98d907f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:19.758516 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd369ef-9ff1-47e7-a666-791f98d907f2 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:19.759306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd369ef-9ff1-47e7-a666-791f98d907f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:19.759687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd369ef-9ff1-47e7-a666-791f98d907f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:19.761252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd369ef-9ff1-47e7-a666-791f98d907f2 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:19.764154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:19.769141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0303b9f9-9e06-41fb-bc35-b4ec125a841f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:19.771855 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0303b9f9-9e06-41fb-bc35-b4ec125a841f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:19.786491 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 397/798] 10.222.0.43 () {56 vars in 1149 bytes} [Wed Jul 20 19:11:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:19.787979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61910178-1e31-4fee-8fef-cb8c304f31ea None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:19.788374 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61910178-1e31-4fee-8fef-cb8c304f31ea None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:19.788723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61910178-1e31-4fee-8fef-cb8c304f31ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:19.792216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-0303b9f9-9e06-41fb-bc35-b4ec125a841f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:19.792216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:19.792216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:19.792216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:19.792216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:19.792216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:19.792216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:19.792216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:19.792216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:19.792216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:19.792216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:19.792216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:19.792216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:19.792216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:19.792216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:19.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:19.794051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:19.794051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:19.794051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:19.794051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:19.800646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 402/799] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:19.802443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303b9f9-9e06-41fb-bc35-b4ec125a841f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:19.808341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-61910178-1e31-4fee-8fef-cb8c304f31ea None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:19.809668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-61910178-1e31-4fee-8fef-cb8c304f31ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:19.882936 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 398/800] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:19.889242 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61910178-1e31-4fee-8fef-cb8c304f31ea None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:19.910419 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303b9f9-9e06-41fb-bc35-b4ec125a841f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:19.948366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f42b05-7ac4-4c92-b771-1871f1c0f70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:19.949172 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f42b05-7ac4-4c92-b771-1871f1c0f70a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:19.949455 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f42b05-7ac4-4c92-b771-1871f1c0f70a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:19.949729 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f42b05-7ac4-4c92-b771-1871f1c0f70a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:19.961099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f42b05-7ac4-4c92-b771-1871f1c0f70a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:19.962656 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f42b05-7ac4-4c92-b771-1871f1c0f70a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:19.981703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-e7f42b05-7ac4-4c92-b771-1871f1c0f70a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:19.981703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:19.981703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:19.981703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:19.981703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:19.981703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:19.981703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:19.981703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:19.981703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:19.981703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:19.981703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:19.981703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:19.981703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:19.981703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:19.981703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:19.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:19.983529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:19.983529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:19.983529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:19.983529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:19.987704 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 403/801] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:19.988597 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f42b05-7ac4-4c92-b771-1871f1c0f70a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.007331 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:20.017434 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61910178-1e31-4fee-8fef-cb8c304f31ea None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.042502 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:20.055492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307d2a7-945f-4a03-a7b7-b2e75f45186c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.056057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307d2a7-945f-4a03-a7b7-b2e75f45186c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.056478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307d2a7-945f-4a03-a7b7-b2e75f45186c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.056752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307d2a7-945f-4a03-a7b7-b2e75f45186c None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.057477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307d2a7-945f-4a03-a7b7-b2e75f45186c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:20.058060 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307d2a7-945f-4a03-a7b7-b2e75f45186c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:20.059401 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307d2a7-945f-4a03-a7b7-b2e75f45186c None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:20.072215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 399/802] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:20.073033 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307d2a7-945f-4a03-a7b7-b2e75f45186c None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.106412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f42b05-7ac4-4c92-b771-1871f1c0f70a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.143883 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bef56-af27-4e50-b5dc-a3156ee10a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.145109 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bef56-af27-4e50-b5dc-a3156ee10a05 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.145643 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bef56-af27-4e50-b5dc-a3156ee10a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.146149 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bef56-af27-4e50-b5dc-a3156ee10a05 None admin] PATH_INFO: `/v3/services/network` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.147113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bef56-af27-4e50-b5dc-a3156ee10a05 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:20.148516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bef56-af27-4e50-b5dc-a3156ee10a05 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-df5bef56-af27-4e50-b5dc-a3156ee10a05 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:20.161647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:20.162638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:20.162638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:20.162638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:20.162638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:20.162638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:20.162638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:20.162638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:20.162638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:20.162638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:20.162638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:20.162638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:20.162638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:20.162638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:20.162638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:20.162638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:20.162638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:20.163476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:20.163476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:20.163476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:20.163476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:20.163476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:20.163476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:20.163476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:20.163476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:20.163476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:20.163476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:20.163476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:20.163476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:20.163476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:11:20.163476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:20.163476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 404/803] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:19 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:20.164266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa49953-01b9-4847-a978-ee25f07e2eeb None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.164894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa49953-01b9-4847-a978-ee25f07e2eeb None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.164894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa49953-01b9-4847-a978-ee25f07e2eeb None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.170164 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 405/804] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:20.170164 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa49953-01b9-4847-a978-ee25f07e2eeb None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.183393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307d2a7-945f-4a03-a7b7-b2e75f45186c None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.217633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae2bde-f155-41a2-9cb2-34f2159b8171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.218300 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae2bde-f155-41a2-9cb2-34f2159b8171 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.218578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae2bde-f155-41a2-9cb2-34f2159b8171 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.218859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae2bde-f155-41a2-9cb2-34f2159b8171 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.219587 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae2bde-f155-41a2-9cb2-34f2159b8171 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:20.219895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae2bde-f155-41a2-9cb2-34f2159b8171 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:20.221288 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae2bde-f155-41a2-9cb2-34f2159b8171 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:20.234112 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 400/805] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:20.239923 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae2bde-f155-41a2-9cb2-34f2159b8171 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.251256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:20.278014 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa49953-01b9-4847-a978-ee25f07e2eeb None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.286534 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:20.314794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837b4ac-54c4-4572-a1da-a94f0a6ecad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.315578 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837b4ac-54c4-4572-a1da-a94f0a6ecad9 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.315854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837b4ac-54c4-4572-a1da-a94f0a6ecad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.316120 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837b4ac-54c4-4572-a1da-a94f0a6ecad9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.322675 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837b4ac-54c4-4572-a1da-a94f0a6ecad9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:20.324104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837b4ac-54c4-4572-a1da-a94f0a6ecad9 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:20.338078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-f837b4ac-54c4-4572-a1da-a94f0a6ecad9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:20.338078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:20.338078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:20.338078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:20.338078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:20.338078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:20.338078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:20.338078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:20.338078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:20.338078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:20.338078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:20.338078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:20.338078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:20.338078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:20.338078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:20.339090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:20.339946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:20.339946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:20.339946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:20.339946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:20.339946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:20.339946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:20.339946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:20.339946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:20.339946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 406/806] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:20.340449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29437b27-d2c3-420b-ac97-0d8ee2d89abe None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.340886 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29437b27-d2c3-420b-ac97-0d8ee2d89abe None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.341165 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29437b27-d2c3-420b-ac97-0d8ee2d89abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.349751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae2bde-f155-41a2-9cb2-34f2159b8171 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.360254 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-29437b27-d2c3-420b-ac97-0d8ee2d89abe None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:20.361464 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-29437b27-d2c3-420b-ac97-0d8ee2d89abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:20.384644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8c258-6a2d-4b97-b218-fef8b3af4e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.385311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8c258-6a2d-4b97-b218-fef8b3af4e7f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.385589 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8c258-6a2d-4b97-b218-fef8b3af4e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.385858 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8c258-6a2d-4b97-b218-fef8b3af4e7f None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.386565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8c258-6a2d-4b97-b218-fef8b3af4e7f None admin] RBAC: Adding query filter params (name=network) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:20.386863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8c258-6a2d-4b97-b218-fef8b3af4e7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:20.388195 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8c258-6a2d-4b97-b218-fef8b3af4e7f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:20.392617 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 401/807] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:20.393396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8c258-6a2d-4b97-b218-fef8b3af4e7f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.434198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 407/808] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:20.435839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38813c5-29e3-4e8f-95b7-42a61d8dc461 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.436129 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38813c5-29e3-4e8f-95b7-42a61d8dc461 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.436564 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38813c5-29e3-4e8f-95b7-42a61d8dc461 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.437983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 408/809] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:20.438533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38813c5-29e3-4e8f-95b7-42a61d8dc461 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.500788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8c258-6a2d-4b97-b218-fef8b3af4e7f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.509878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:20.517283 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:20.536456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a20ed-3cca-4c4e-a914-b611c95356e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.537252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a20ed-3cca-4c4e-a914-b611c95356e8 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.537536 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a20ed-3cca-4c4e-a914-b611c95356e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.537819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a20ed-3cca-4c4e-a914-b611c95356e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.544549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a20ed-3cca-4c4e-a914-b611c95356e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:20.546005 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a20ed-3cca-4c4e-a914-b611c95356e8 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:20.553520 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38813c5-29e3-4e8f-95b7-42a61d8dc461 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.559794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-001a20ed-3cca-4c4e-a914-b611c95356e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:20.559794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:20.559794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:20.559794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:20.559794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:20.559794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:20.559794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:20.559794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:20.559794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:20.559794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:20.559794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:20.559794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:20.559794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:20.559794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:20.559794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:20.561069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:20.562186 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:20.562186 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:20.562186 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:20.562186 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:20.562186 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:20.562186 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:20.562186 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:20.562186 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:20.562186 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 402/810] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:20.564043 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a20ed-3cca-4c4e-a914-b611c95356e8 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.589331 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387ad9c-b983-460e-92a5-5bb18548adc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.590221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387ad9c-b983-460e-92a5-5bb18548adc1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.590504 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387ad9c-b983-460e-92a5-5bb18548adc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.590784 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387ad9c-b983-460e-92a5-5bb18548adc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.591574 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387ad9c-b983-460e-92a5-5bb18548adc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:20.591878 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387ad9c-b983-460e-92a5-5bb18548adc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:20.593331 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387ad9c-b983-460e-92a5-5bb18548adc1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:20.605135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 409/811] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:20.606334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a09d3-6487-4b3a-a4b5-0f574a669856 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.606619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a09d3-6487-4b3a-a4b5-0f574a669856 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.607203 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a09d3-6487-4b3a-a4b5-0f574a669856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.622269 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d13a09d3-6487-4b3a-a4b5-0f574a669856 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:20.623457 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d13a09d3-6487-4b3a-a4b5-0f574a669856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:20.664816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a20ed-3cca-4c4e-a914-b611c95356e8 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.688262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 410/812] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:20.689109 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a09d3-6487-4b3a-a4b5-0f574a669856 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.711489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6b227-b55d-4f30-80ec-b0363f21c70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.712317 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6b227-b55d-4f30-80ec-b0363f21c70a None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.712707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6b227-b55d-4f30-80ec-b0363f21c70a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.713082 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6b227-b55d-4f30-80ec-b0363f21c70a None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.713904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6b227-b55d-4f30-80ec-b0363f21c70a None admin] RBAC: Adding query filter params (type=network) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:20.714285 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6b227-b55d-4f30-80ec-b0363f21c70a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:20.715647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6b227-b55d-4f30-80ec-b0363f21c70a None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:20.720762 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 403/813] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:20.721530 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6b227-b55d-4f30-80ec-b0363f21c70a None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.746865 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:20.793278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a09d3-6487-4b3a-a4b5-0f574a669856 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.829322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cf817-4a7d-442a-b48c-dc5d084e4ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.830122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cf817-4a7d-442a-b48c-dc5d084e4ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.830399 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cf817-4a7d-442a-b48c-dc5d084e4ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.830668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cf817-4a7d-442a-b48c-dc5d084e4ac1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.845142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789cf817-4a7d-442a-b48c-dc5d084e4ac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:20.853196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-789cf817-4a7d-442a-b48c-dc5d084e4ac1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:20.853196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:20.853196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:20.853196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:20.853196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:20.853196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:20.853196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:20.853196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:20.853196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:20.853196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:20.853196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:20.853196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:20.853196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:20.853196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:20.853196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:20.853196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:20.854309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:20.854309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:20.854309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:20.854309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:20.854309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:20.854309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:20.854309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:20.854309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:20.854309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:20.854309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:20.854309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:20.854309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:20.854309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:20.854309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:20.854309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:20.854309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:20.855315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:20.855315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:20.855315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:20.855315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:20.855315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:20.855315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:20.855315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:20.855315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:20.855315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:20.855315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:20.855315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:20.855315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:20.855315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 411/814] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:20.855315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cf817-4a7d-442a-b48c-dc5d084e4ac1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.872537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:20.946098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6b227-b55d-4f30-80ec-b0363f21c70a None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.953357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cf817-4a7d-442a-b48c-dc5d084e4ac1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.969586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:20.980655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20080484-93a0-4014-b913-d890651a99c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.981316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20080484-93a0-4014-b913-d890651a99c7 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.981593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20080484-93a0-4014-b913-d890651a99c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.981884 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20080484-93a0-4014-b913-d890651a99c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.982644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20080484-93a0-4014-b913-d890651a99c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:20.982946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20080484-93a0-4014-b913-d890651a99c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:20.984364 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20080484-93a0-4014-b913-d890651a99c7 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:20.987057 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ce034-ef1e-48f7-aa20-eb7927b0224c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.987696 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ce034-ef1e-48f7-aa20-eb7927b0224c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.987976 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ce034-ef1e-48f7-aa20-eb7927b0224c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.988246 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ce034-ef1e-48f7-aa20-eb7927b0224c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.989156 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ce034-ef1e-48f7-aa20-eb7927b0224c None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=267b178236e046e4bc694b2da6f86eb0) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:20.989465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ce034-ef1e-48f7-aa20-eb7927b0224c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:20.990867 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ce034-ef1e-48f7-aa20-eb7927b0224c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:20.995494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 404/815] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:20.996266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20080484-93a0-4014-b913-d890651a99c7 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.004972 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 412/816] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:20 2022] GET /identity/v3/role_assignments?user.id=267b178236e046e4bc694b2da6f86eb0&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:21.004972 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ce034-ef1e-48f7-aa20-eb7927b0224c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.103944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:21.113311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20080484-93a0-4014-b913-d890651a99c7 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.125616 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ce034-ef1e-48f7-aa20-eb7927b0224c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.155628 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6315636-82b5-462f-8cb0-b4870aec1b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.156499 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6315636-82b5-462f-8cb0-b4870aec1b12 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.157407 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6315636-82b5-462f-8cb0-b4870aec1b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.157407 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6315636-82b5-462f-8cb0-b4870aec1b12 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.157697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6315636-82b5-462f-8cb0-b4870aec1b12 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.159114 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6315636-82b5-462f-8cb0-b4870aec1b12 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.165717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b5a00-a9be-446b-a913-a357f634cddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.166516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b5a00-a9be-446b-a913-a357f634cddf None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.166675 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b5a00-a9be-446b-a913-a357f634cddf None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.166925 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b5a00-a9be-446b-a913-a357f634cddf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.167845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b5a00-a9be-446b-a913-a357f634cddf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.169285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b5a00-a9be-446b-a913-a357f634cddf None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.172127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-a6315636-82b5-462f-8cb0-b4870aec1b12 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:11:21.172127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:21.172127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:21.172127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:21.172127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:21.172127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:21.172127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:21.172127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:21.172127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:11:21.172127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:21.172127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:21.172127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:21.172127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:21.172127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:21.172127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:21.172127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:21.173152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:21.173152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:21.173152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:21.173152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:21.173152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:21.173152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:21.173152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:21.173152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:21.173152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:21.173152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:21.173152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:21.173152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:21.173152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:21.173152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:21.173152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:21.173152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:21.173984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:21.173984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:21.173984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:21.173984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:21.173984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:21.173984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:21.173984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:21.173984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:21.173984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:21.173984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:21.173984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:21.173984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:21.173984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:21.173984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:11:21.173984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:21.173984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 405/817] 10.222.0.43 () {56 vars in 1129 bytes} [Wed Jul 20 19:11:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:21.174800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6315636-82b5-462f-8cb0-b4870aec1b12 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.197418 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 413/818] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jul 20 19:11:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:21.198312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b5a00-a9be-446b-a913-a357f634cddf None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.244677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:21.293900 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6315636-82b5-462f-8cb0-b4870aec1b12 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.349780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b5a00-a9be-446b-a913-a357f634cddf None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.353257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef791808-7d4a-467d-bb04-3e3b57590dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.354406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef791808-7d4a-467d-bb04-3e3b57590dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.354725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef791808-7d4a-467d-bb04-3e3b57590dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.356048 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef791808-7d4a-467d-bb04-3e3b57590dc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.356048 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef791808-7d4a-467d-bb04-3e3b57590dc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:21.356461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef791808-7d4a-467d-bb04-3e3b57590dc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.357932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef791808-7d4a-467d-bb04-3e3b57590dc5 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.372328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 406/819] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:21.372328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef791808-7d4a-467d-bb04-3e3b57590dc5 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.388754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:21.398682 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55f036-aca8-4edc-82e3-7ecc9ccc4280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.399934 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55f036-aca8-4edc-82e3-7ecc9ccc4280 None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.400535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55f036-aca8-4edc-82e3-7ecc9ccc4280 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.401338 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55f036-aca8-4edc-82e3-7ecc9ccc4280 None admin] PATH_INFO: `/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/2d514236db644010859fc18250f5894e/roles/7fe62c5c771d41c28f790fade1df9357` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.423181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55f036-aca8-4edc-82e3-7ecc9ccc4280 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e2a50ebbb48417c911df499d98a0157, user_id=2d514236db644010859fc18250f5894e, role_id=7fe62c5c771d41c28f790fade1df9357)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.424395 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55f036-aca8-4edc-82e3-7ecc9ccc4280 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.443723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 414/820] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:11:21 2022] PUT /identity/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/2d514236db644010859fc18250f5894e/roles/7fe62c5c771d41c28f790fade1df9357 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:11:21.448754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55f036-aca8-4edc-82e3-7ecc9ccc4280 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.497686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef791808-7d4a-467d-bb04-3e3b57590dc5 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.539752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c2b3a-15cc-4ec3-9179-28c75a6664a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.540595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c2b3a-15cc-4ec3-9179-28c75a6664a4 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.540858 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c2b3a-15cc-4ec3-9179-28c75a6664a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.541132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c2b3a-15cc-4ec3-9179-28c75a6664a4 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.541893 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c2b3a-15cc-4ec3-9179-28c75a6664a4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:21.542317 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c2b3a-15cc-4ec3-9179-28c75a6664a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.543713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c2b3a-15cc-4ec3-9179-28c75a6664a4 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.549228 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 407/821] 10.222.0.43 () {56 vars in 1124 bytes} [Wed Jul 20 19:11:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:21.555223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c2b3a-15cc-4ec3-9179-28c75a6664a4 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.578405 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:21.608223 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55f036-aca8-4edc-82e3-7ecc9ccc4280 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.651260 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f506e38-ca20-47ae-ad45-1aa910b84b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.652155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f506e38-ca20-47ae-ad45-1aa910b84b0c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.652524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f506e38-ca20-47ae-ad45-1aa910b84b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.652835 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f506e38-ca20-47ae-ad45-1aa910b84b0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.655270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:21.666703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f506e38-ca20-47ae-ad45-1aa910b84b0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.668168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f506e38-ca20-47ae-ad45-1aa910b84b0c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.688225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c2b3a-15cc-4ec3-9179-28c75a6664a4 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.689134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-7f506e38-ca20-47ae-ad45-1aa910b84b0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:21.689134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:21.689134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:21.689134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:21.689134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:21.689134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:21.689134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:21.689134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:21.689134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:21.689134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:21.689134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:21.689134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:21.689134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:21.689134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:21.689134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:21.690220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:21.691302 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:21.691302 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:21.691302 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:21.691302 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:21.691302 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 415/822] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:21.696036 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f506e38-ca20-47ae-ad45-1aa910b84b0c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.732959 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820a78c-00c6-44eb-bbe1-bba5262911a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.733835 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820a78c-00c6-44eb-bbe1-bba5262911a2 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.734178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820a78c-00c6-44eb-bbe1-bba5262911a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.735232 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820a78c-00c6-44eb-bbe1-bba5262911a2 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.735954 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820a78c-00c6-44eb-bbe1-bba5262911a2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:21.736267 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820a78c-00c6-44eb-bbe1-bba5262911a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.737662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820a78c-00c6-44eb-bbe1-bba5262911a2 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.744632 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 408/823] 10.222.0.43 () {56 vars in 1124 bytes} [Wed Jul 20 19:11:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:21.750513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820a78c-00c6-44eb-bbe1-bba5262911a2 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.809473 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f506e38-ca20-47ae-ad45-1aa910b84b0c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.844810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08933f70-6532-475c-85b9-f27b609b20ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.845348 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08933f70-6532-475c-85b9-f27b609b20ac None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.845629 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08933f70-6532-475c-85b9-f27b609b20ac None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.845902 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08933f70-6532-475c-85b9-f27b609b20ac None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.846627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08933f70-6532-475c-85b9-f27b609b20ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:21.846932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08933f70-6532-475c-85b9-f27b609b20ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.849644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08933f70-6532-475c-85b9-f27b609b20ac None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.850383 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:21.872499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 416/824] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:21.875838 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08933f70-6532-475c-85b9-f27b609b20ac None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.897695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820a78c-00c6-44eb-bbe1-bba5262911a2 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.945480 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eaf569-4768-49d3-b979-e666faab40fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.946575 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaf569-4768-49d3-b979-e666faab40fe None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.947101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaf569-4768-49d3-b979-e666faab40fe None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.947572 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaf569-4768-49d3-b979-e666faab40fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.948564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eaf569-4768-49d3-b979-e666faab40fe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.950168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eaf569-4768-49d3-b979-e666faab40fe None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.968682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:21.982250 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 409/825] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jul 20 19:11:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:22.140162 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08933f70-6532-475c-85b9-f27b609b20ac None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:22.181495 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274dee68-26c5-43fd-9fd5-c90b83809516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:22.182653 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274dee68-26c5-43fd-9fd5-c90b83809516 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:22.183401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274dee68-26c5-43fd-9fd5-c90b83809516 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:22.183892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274dee68-26c5-43fd-9fd5-c90b83809516 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:22.191736 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274dee68-26c5-43fd-9fd5-c90b83809516 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:22.193239 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274dee68-26c5-43fd-9fd5-c90b83809516 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:22.208613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-274dee68-26c5-43fd-9fd5-c90b83809516 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:11:22.208613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:22.208613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:22.208613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:22.208613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:22.208613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:22.208613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:22.208613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:22.208613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:22.208613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:22.208613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:22.208613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:22.208613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:22.208613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:22.208613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:22.208613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:22.209605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:22.210522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:22.210522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:22.210522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:22.210522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:22.210522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:22.210522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:22.210522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:22.210522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:22.210522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:11:22.210522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:22.210522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 417/826] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:22.217429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eaf569-4768-49d3-b979-e666faab40fe None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:22.325542 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eaf569-4768-49d3-b979-e666faab40fe None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:22.363213 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574dd3b-bb55-49cb-a874-1f8f869e9744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:22.363960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574dd3b-bb55-49cb-a874-1f8f869e9744 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:22.364247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574dd3b-bb55-49cb-a874-1f8f869e9744 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:22.364554 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574dd3b-bb55-49cb-a874-1f8f869e9744 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:22.365403 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574dd3b-bb55-49cb-a874-1f8f869e9744 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:22.365712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574dd3b-bb55-49cb-a874-1f8f869e9744 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:22.367125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574dd3b-bb55-49cb-a874-1f8f869e9744 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:22.386011 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 410/827] 10.222.0.43 () {56 vars in 1135 bytes} [Wed Jul 20 19:11:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:22.391721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274dee68-26c5-43fd-9fd5-c90b83809516 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:22.448513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:22.502137 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274dee68-26c5-43fd-9fd5-c90b83809516 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:22.541268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cd83a-9020-43ca-8a0a-8bfd0cb2d83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:22.542055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cd83a-9020-43ca-8a0a-8bfd0cb2d83e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:22.542336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cd83a-9020-43ca-8a0a-8bfd0cb2d83e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:22.542610 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cd83a-9020-43ca-8a0a-8bfd0cb2d83e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:22.553874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6cd83a-9020-43ca-8a0a-8bfd0cb2d83e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:22.555316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6cd83a-9020-43ca-8a0a-8bfd0cb2d83e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:22.573452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-bc6cd83a-9020-43ca-8a0a-8bfd0cb2d83e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:22.573452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:22.573452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:22.573452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:22.573452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:22.573452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:22.573452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:22.573452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:22.573452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:22.573452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:22.573452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:22.573452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:22.573452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:22.573452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:22.573452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:22.574357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:22.575250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:22.575250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:22.575250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:22.575250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:22.575250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 418/828] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:22.580629 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574dd3b-bb55-49cb-a874-1f8f869e9744 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:22.641708 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:22.688507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574dd3b-bb55-49cb-a874-1f8f869e9744 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:22.724597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f15f7a-6428-41a5-b50b-5b85f9e88a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:22.725267 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f15f7a-6428-41a5-b50b-5b85f9e88a47 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:22.725546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f15f7a-6428-41a5-b50b-5b85f9e88a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:22.725813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f15f7a-6428-41a5-b50b-5b85f9e88a47 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:22.726540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f15f7a-6428-41a5-b50b-5b85f9e88a47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:22.726844 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f15f7a-6428-41a5-b50b-5b85f9e88a47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:22.728262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f15f7a-6428-41a5-b50b-5b85f9e88a47 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:22.741042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 411/829] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:22.746519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cd83a-9020-43ca-8a0a-8bfd0cb2d83e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:22.856669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cd83a-9020-43ca-8a0a-8bfd0cb2d83e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:22.858177 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:22.894462 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4e6b1-b6aa-46ee-872a-6f5c1be581ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:22.895286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4e6b1-b6aa-46ee-872a-6f5c1be581ce None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:22.895560 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4e6b1-b6aa-46ee-872a-6f5c1be581ce None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:22.895842 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4e6b1-b6aa-46ee-872a-6f5c1be581ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:22.902722 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4e6b1-b6aa-46ee-872a-6f5c1be581ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:22.904167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4e6b1-b6aa-46ee-872a-6f5c1be581ce None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:22.917823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-2ff4e6b1-b6aa-46ee-872a-6f5c1be581ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:22.917823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:22.917823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:22.917823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:22.917823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:22.917823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:22.917823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:22.917823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:22.917823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:22.917823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:22.917823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:22.917823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:22.917823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:22.917823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:22.917823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:22.920263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:22.922708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:22.922708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:22.922708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:22.922708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:22.922708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:22.922708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:22.922708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:22.922708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:22.922708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 419/830] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:22.928318 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f15f7a-6428-41a5-b50b-5b85f9e88a47 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:23.030132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f15f7a-6428-41a5-b50b-5b85f9e88a47 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:23.046999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:23.068926 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa626bb-27b3-4336-a917-660d75df4de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:23.069589 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa626bb-27b3-4336-a917-660d75df4de6 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.069869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa626bb-27b3-4336-a917-660d75df4de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.070148 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa626bb-27b3-4336-a917-660d75df4de6 None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.070936 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa626bb-27b3-4336-a917-660d75df4de6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:23.071243 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa626bb-27b3-4336-a917-660d75df4de6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:23.072668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa626bb-27b3-4336-a917-660d75df4de6 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:23.085311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 412/831] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:23.091260 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4e6b1-b6aa-46ee-872a-6f5c1be581ce None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:23.184863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62690282-4356-4f12-a896-6c03a994017d None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.185217 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62690282-4356-4f12-a896-6c03a994017d None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.185444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62690282-4356-4f12-a896-6c03a994017d None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.190669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 413/832] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:23.196362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bb1ef-ee20-427e-b813-ee5ff1fda9e9 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.196362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bb1ef-ee20-427e-b813-ee5ff1fda9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.196362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bb1ef-ee20-427e-b813-ee5ff1fda9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.203830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4e6b1-b6aa-46ee-872a-6f5c1be581ce None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:23.214440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-3e5bb1ef-ee20-427e-b813-ee5ff1fda9e9 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:23.216450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-3e5bb1ef-ee20-427e-b813-ee5ff1fda9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:23.240571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df2286-73a7-4d7b-9c8a-c004bcfda81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:23.241578 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df2286-73a7-4d7b-9c8a-c004bcfda81a None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.241929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df2286-73a7-4d7b-9c8a-c004bcfda81a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.242277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df2286-73a7-4d7b-9c8a-c004bcfda81a None admin] PATH_INFO: `/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/1c944eab8a8948749bb8375787216b09/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.244377 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df2286-73a7-4d7b-9c8a-c004bcfda81a None admin] RBAC: Authorizing `identity:create_grant(project_id=5e2a50ebbb48417c911df499d98a0157, user_id=1c944eab8a8948749bb8375787216b09, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:23.245895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df2286-73a7-4d7b-9c8a-c004bcfda81a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:23.263603 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 420/833] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:11:23 2022] PUT /identity/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/1c944eab8a8948749bb8375787216b09/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:11:23.312035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 414/834] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:23.339762 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df2286-73a7-4d7b-9c8a-c004bcfda81a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:23.373966 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:23.458974 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df2286-73a7-4d7b-9c8a-c004bcfda81a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:23.494823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef97127-e680-43db-bcae-49a3125edbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:23.495630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef97127-e680-43db-bcae-49a3125edbef None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.495905 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef97127-e680-43db-bcae-49a3125edbef None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.496173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef97127-e680-43db-bcae-49a3125edbef None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.496934 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef97127-e680-43db-bcae-49a3125edbef None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:23.498209 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef97127-e680-43db-bcae-49a3125edbef None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:23.510572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-eef97127-e680-43db-bcae-49a3125edbef None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:11:23.510572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:23.510572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:23.510572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:23.510572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:23.510572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:23.510572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:23.510572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:23.510572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:11:23.510572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:23.510572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:23.510572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:23.510572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:23.510572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:23.510572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:23.510572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:23.511546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:23.511546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:23.511546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:23.511546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:23.511546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:23.511546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:23.511546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:23.511546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:23.511546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:23.511546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:23.511546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:23.511546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:23.511546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:23.511546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:23.511546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:23.511546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:23.512665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:23.512665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:23.512665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:23.512665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:23.512665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:23.512665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:23.512665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:23.512665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:23.512665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:23.512665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:23.512665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:23.512665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:23.512665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:23.512665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:11:23.512665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:23.512665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 421/835] 10.222.0.43 () {56 vars in 1132 bytes} [Wed Jul 20 19:11:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:23.518091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5bb1ef-ee20-427e-b813-ee5ff1fda9e9 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:23.622889 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5bb1ef-ee20-427e-b813-ee5ff1fda9e9 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:23.631361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:23.661146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a847623-9493-4ae6-a82e-170044c6d93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:23.661832 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a847623-9493-4ae6-a82e-170044c6d93a None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.662113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a847623-9493-4ae6-a82e-170044c6d93a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.662383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a847623-9493-4ae6-a82e-170044c6d93a None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.663093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a847623-9493-4ae6-a82e-170044c6d93a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:23.663392 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a847623-9493-4ae6-a82e-170044c6d93a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:23.664776 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a847623-9493-4ae6-a82e-170044c6d93a None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:23.669731 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 415/836] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:23.675023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef97127-e680-43db-bcae-49a3125edbef None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:23.765432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:23.782272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef97127-e680-43db-bcae-49a3125edbef None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:23.818134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df3a5b-64a7-421b-9ba0-0696caaa2209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:23.819140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df3a5b-64a7-421b-9ba0-0696caaa2209 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.819638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df3a5b-64a7-421b-9ba0-0696caaa2209 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.820107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df3a5b-64a7-421b-9ba0-0696caaa2209 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.821056 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df3a5b-64a7-421b-9ba0-0696caaa2209 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:23.821555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df3a5b-64a7-421b-9ba0-0696caaa2209 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:23.823100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df3a5b-64a7-421b-9ba0-0696caaa2209 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:23.828489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 422/837] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:23 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:23.839665 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b06f6c-131f-4427-925c-c9e057dd8c68 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.840007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b06f6c-131f-4427-925c-c9e057dd8c68 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.840219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b06f6c-131f-4427-925c-c9e057dd8c68 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.842027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 416/838] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:23.847807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e42a0-4ed4-4d0f-b494-fcbe8d09708b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.848684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e42a0-4ed4-4d0f-b494-fcbe8d09708b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.848982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e42a0-4ed4-4d0f-b494-fcbe8d09708b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.868616 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-518e42a0-4ed4-4d0f-b494-fcbe8d09708b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:23.870100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-518e42a0-4ed4-4d0f-b494-fcbe8d09708b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:23.954059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 423/839] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:23.978346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b06f6c-131f-4427-925c-c9e057dd8c68 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.091842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b06f6c-131f-4427-925c-c9e057dd8c68 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.105816 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b44b96-f8ab-4a3b-9258-b0f94dafbf1e None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.106114 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b44b96-f8ab-4a3b-9258-b0f94dafbf1e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.106503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b44b96-f8ab-4a3b-9258-b0f94dafbf1e None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.108116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 424/840] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:24.113315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c30f57-384d-45d6-a074-e2b2e6b79898 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.113627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c30f57-384d-45d6-a074-e2b2e6b79898 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.114465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c30f57-384d-45d6-a074-e2b2e6b79898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.128919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ed534-ab56-49ac-bf8f-e66392ce4dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.129756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ed534-ab56-49ac-bf8f-e66392ce4dab None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.130033 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ed534-ab56-49ac-bf8f-e66392ce4dab None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.130305 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ed534-ab56-49ac-bf8f-e66392ce4dab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.131429 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-08c30f57-384d-45d6-a074-e2b2e6b79898 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:24.132671 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-08c30f57-384d-45d6-a074-e2b2e6b79898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:24.141261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ed534-ab56-49ac-bf8f-e66392ce4dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:24.149531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-e92ed534-ab56-49ac-bf8f-e66392ce4dab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:11:24.149531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:24.149531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:24.149531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:24.149531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:24.149531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:24.149531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:24.149531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:24.149531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:24.149531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:24.149531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:24.149531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:24.149531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:24.149531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:24.149531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:24.149531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:24.150439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:24.150439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:24.150439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:24.150439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:24.150439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:24.150439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:24.150439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:24.150439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:24.150439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:24.150439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:24.150439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:24.150439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:24.150439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:24.150439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:24.150439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:24.150439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:24.151290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:24.151290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:24.151290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:24.151290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:24.151290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:24.151290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:24.151290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:24.151290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:24.151290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:24.151290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:24.151290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:11:24.151290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:24.151290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 417/841] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:11:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:24.156585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ed534-ab56-49ac-bf8f-e66392ce4dab None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.204789 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:24.228329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 425/842] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:24.242899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c30f57-384d-45d6-a074-e2b2e6b79898 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.257967 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:24.276207 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ed534-ab56-49ac-bf8f-e66392ce4dab None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.319256 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461a913-c492-49c0-bf91-7fe76c507d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.320807 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461a913-c492-49c0-bf91-7fe76c507d4a None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.321383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461a913-c492-49c0-bf91-7fe76c507d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.321975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461a913-c492-49c0-bf91-7fe76c507d4a None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.323220 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461a913-c492-49c0-bf91-7fe76c507d4a None admin] RBAC: Adding query filter params (name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:24.323984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461a913-c492-49c0-bf91-7fe76c507d4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:24.325548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461a913-c492-49c0-bf91-7fe76c507d4a None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:24.337327 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 418/843] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:11:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:24.343437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461a913-c492-49c0-bf91-7fe76c507d4a None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.373720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c30f57-384d-45d6-a074-e2b2e6b79898 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.420875 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0a9db-1776-4da4-a3d0-0bf9d0a87f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.421672 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0a9db-1776-4da4-a3d0-0bf9d0a87f8f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.421949 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0a9db-1776-4da4-a3d0-0bf9d0a87f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.422218 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0a9db-1776-4da4-a3d0-0bf9d0a87f8f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.422877 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0a9db-1776-4da4-a3d0-0bf9d0a87f8f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:24.424233 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0a9db-1776-4da4-a3d0-0bf9d0a87f8f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-e4b0a9db-1776-4da4-a3d0-0bf9d0a87f8f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:24.443145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:24.444144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:24.444144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:24.444144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:24.444144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:24.444144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:24.444144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:24.444144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:24.444144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:24.444144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:24.444144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:24.444144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:24.444144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:24.444144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:24.444144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:24.444144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:24.444144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:24.444998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:24.444998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:24.444998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:24.444998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:24.444998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:24.444998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:24.444998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:24.444998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:24.444998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:24.444998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:24.444998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:24.444998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:24.444998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:11:24.444998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:24.446458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 426/844] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:11:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:24.456465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0a9db-1776-4da4-a3d0-0bf9d0a87f8f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.461270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:24.471273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461a913-c492-49c0-bf91-7fe76c507d4a None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.510516 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6136ed-ad52-4876-909f-d0a14b24cddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.511699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6136ed-ad52-4876-909f-d0a14b24cddc None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.512130 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6136ed-ad52-4876-909f-d0a14b24cddc None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.512723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6136ed-ad52-4876-909f-d0a14b24cddc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.525578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6136ed-ad52-4876-909f-d0a14b24cddc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:24.527013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6136ed-ad52-4876-909f-d0a14b24cddc None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:24.546548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-fc6136ed-ad52-4876-909f-d0a14b24cddc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:24.546548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:24.546548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:24.546548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:24.546548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:24.546548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:24.546548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:24.546548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:24.546548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:24.546548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:24.546548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:24.546548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:24.546548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:24.546548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:24.546548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:24.547506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:24.548425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:24.548425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:24.548425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:24.548425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:24.548425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 419/845] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:24.549219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:24.553441 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6136ed-ad52-4876-909f-d0a14b24cddc None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.579454 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0a9db-1776-4da4-a3d0-0bf9d0a87f8f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.615966 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19761e-efbd-4880-ae77-aec880c94b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.616662 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19761e-efbd-4880-ae77-aec880c94b44 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.616946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19761e-efbd-4880-ae77-aec880c94b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.617221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19761e-efbd-4880-ae77-aec880c94b44 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.618073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19761e-efbd-4880-ae77-aec880c94b44 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:24.618254 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19761e-efbd-4880-ae77-aec880c94b44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:24.619645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19761e-efbd-4880-ae77-aec880c94b44 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:24.629814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 427/846] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:11:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:24.630778 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19761e-efbd-4880-ae77-aec880c94b44 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.680146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6136ed-ad52-4876-909f-d0a14b24cddc None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.715540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c2e2f-f3b2-4ebb-959c-308a56afa5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.716125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c2e2f-f3b2-4ebb-959c-308a56afa5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.716437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c2e2f-f3b2-4ebb-959c-308a56afa5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.716715 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c2e2f-f3b2-4ebb-959c-308a56afa5d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.717437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c2e2f-f3b2-4ebb-959c-308a56afa5d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:24.717745 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c2e2f-f3b2-4ebb-959c-308a56afa5d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:24.719090 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c2e2f-f3b2-4ebb-959c-308a56afa5d5 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:24.731203 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 420/847] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:24.736300 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c2e2f-f3b2-4ebb-959c-308a56afa5d5 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.740080 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:24.750085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19761e-efbd-4880-ae77-aec880c94b44 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.786799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc529e-f1cd-4c83-8d7b-e40e08da82ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.787879 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc529e-f1cd-4c83-8d7b-e40e08da82ec None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.788496 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc529e-f1cd-4c83-8d7b-e40e08da82ec None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.788974 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc529e-f1cd-4c83-8d7b-e40e08da82ec None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.789989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc529e-f1cd-4c83-8d7b-e40e08da82ec None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:24.790618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc529e-f1cd-4c83-8d7b-e40e08da82ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:24.792177 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc529e-f1cd-4c83-8d7b-e40e08da82ec None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:24.804593 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 428/848] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:11:24 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:24.845506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c2e2f-f3b2-4ebb-959c-308a56afa5d5 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.885512 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8507b-f3d4-407d-84ac-8a64d96e5ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.886345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8507b-f3d4-407d-84ac-8a64d96e5ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.886628 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8507b-f3d4-407d-84ac-8a64d96e5ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.886901 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8507b-f3d4-407d-84ac-8a64d96e5ec1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.895833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8507b-f3d4-407d-84ac-8a64d96e5ec1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:24.897313 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8507b-f3d4-407d-84ac-8a64d96e5ec1 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:24.913697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-0ea8507b-f3d4-407d-84ac-8a64d96e5ec1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:11:24.913697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:24.913697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:24.913697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:24.913697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:24.913697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:24.913697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:24.913697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:24.913697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:24.913697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:24.913697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:24.913697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:24.913697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:24.913697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:24.913697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:24.913697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:24.914687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:24.915625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:24.915625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:24.915625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:24.915625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:24.915625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:24.915625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:24.915625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:24.915625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:24.915625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:11:24.915625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:24.915625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 421/849] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jul 20 19:11:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:24.920793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc529e-f1cd-4c83-8d7b-e40e08da82ec None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.923773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:25.033454 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc529e-f1cd-4c83-8d7b-e40e08da82ec None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:25.072643 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de70c0-5125-43de-b81c-5fac6801de9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:25.073324 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de70c0-5125-43de-b81c-5fac6801de9c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:25.073609 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de70c0-5125-43de-b81c-5fac6801de9c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:25.073885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de70c0-5125-43de-b81c-5fac6801de9c None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:25.074738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de70c0-5125-43de-b81c-5fac6801de9c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:25.075052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de70c0-5125-43de-b81c-5fac6801de9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:25.076514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de70c0-5125-43de-b81c-5fac6801de9c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:25.095074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 429/850] 10.222.0.43 () {56 vars in 1149 bytes} [Wed Jul 20 19:11:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:25.100312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8507b-f3d4-407d-84ac-8a64d96e5ec1 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:25.173291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:25.209305 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8507b-f3d4-407d-84ac-8a64d96e5ec1 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:25.244272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c02aa-b498-49f3-9b12-a15278411c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:25.245362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c02aa-b498-49f3-9b12-a15278411c64 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:25.245816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c02aa-b498-49f3-9b12-a15278411c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:25.246261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c02aa-b498-49f3-9b12-a15278411c64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:25.315785 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c02aa-b498-49f3-9b12-a15278411c64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:25.317661 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c02aa-b498-49f3-9b12-a15278411c64 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:25.380704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:25.420529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 422/851] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:25 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 19:11:25.425606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de70c0-5125-43de-b81c-5fac6801de9c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:25.485241 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149d528-21d9-4d47-a125-c21c84390851 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:25.485527 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149d528-21d9-4d47-a125-c21c84390851 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:25.485836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149d528-21d9-4d47-a125-c21c84390851 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:25.487082 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:25.487488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 423/852] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:25.494835 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf706c2b-7071-4ec4-a6f0-bd05280563e5 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:25.495122 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf706c2b-7071-4ec4-a6f0-bd05280563e5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:25.495398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf706c2b-7071-4ec4-a6f0-bd05280563e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:25.512972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-cf706c2b-7071-4ec4-a6f0-bd05280563e5 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:25.514365 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-cf706c2b-7071-4ec4-a6f0-bd05280563e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:25.537196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de70c0-5125-43de-b81c-5fac6801de9c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:25.575441 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c7d53-4c81-4a9b-ad4b-c1be309483f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:25.576178 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c7d53-4c81-4a9b-ad4b-c1be309483f2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:25.576477 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c7d53-4c81-4a9b-ad4b-c1be309483f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:25.576751 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c7d53-4c81-4a9b-ad4b-c1be309483f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:25.587472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c7d53-4c81-4a9b-ad4b-c1be309483f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:25.588983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c7d53-4c81-4a9b-ad4b-c1be309483f2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:25.596928 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 424/853] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jul 20 19:11:25.611890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-ba5c7d53-4c81-4a9b-ad4b-c1be309483f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:25.611890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:25.611890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:25.611890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:25.611890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:25.611890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:25.611890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:25.611890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:25.611890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:25.611890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:25.611890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:25.611890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:25.611890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:25.611890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:25.611890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:25.612843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:25.614290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:25.614290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:25.614290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:25.614290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:25.614290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 430/854] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:25.621487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf706c2b-7071-4ec4-a6f0-bd05280563e5 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:25.622591 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c7d53-4c81-4a9b-ad4b-c1be309483f2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:25.724122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c7d53-4c81-4a9b-ad4b-c1be309483f2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:25.728330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf706c2b-7071-4ec4-a6f0-bd05280563e5 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:25.732158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:25.738653 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:25.763406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da42a72-3e9c-4135-a0ea-84505eef2e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:25.764461 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da42a72-3e9c-4135-a0ea-84505eef2e3d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:25.764886 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da42a72-3e9c-4135-a0ea-84505eef2e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:25.765332 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da42a72-3e9c-4135-a0ea-84505eef2e3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:25.766754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043482e4-4877-480f-9f06-690d9b7d5000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:25.767742 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043482e4-4877-480f-9f06-690d9b7d5000 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:25.768034 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043482e4-4877-480f-9f06-690d9b7d5000 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:25.768330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043482e4-4877-480f-9f06-690d9b7d5000 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:25.769183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043482e4-4877-480f-9f06-690d9b7d5000 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:25.769501 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043482e4-4877-480f-9f06-690d9b7d5000 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:25.771404 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043482e4-4877-480f-9f06-690d9b7d5000 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:25.779178 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da42a72-3e9c-4135-a0ea-84505eef2e3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:25.788657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 425/855] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:25.789221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-4da42a72-3e9c-4135-a0ea-84505eef2e3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:25.789221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:25.789221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:25.789221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:25.789221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:25.789221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:25.789221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:25.789221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:25.789221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:25.789221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:25.789221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:25.789221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:25.789221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:25.789221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:25.789221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:25.789221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:25.790279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:25.790279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:25.790279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:25.790279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:25.790279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:25.790279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:25.790279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:25.790279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:25.790279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:25.790279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:25.790279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:25.790279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:25.790279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:25.790279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:25.790279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:25.790279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:25.791138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:25.791138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:25.791138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:25.791138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:25.791138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:25.791138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:25.791138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:25.791138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:25.791138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:25.791138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:25.791138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:25.791138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:25.791756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043482e4-4877-480f-9f06-690d9b7d5000 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:25.794884 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 431/856] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:25.804424 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da42a72-3e9c-4135-a0ea-84505eef2e3d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:25.910182 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da42a72-3e9c-4135-a0ea-84505eef2e3d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:25.916111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043482e4-4877-480f-9f06-690d9b7d5000 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:25.945463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b98008-79e7-407a-9a8e-efe503064286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:25.946482 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b98008-79e7-407a-9a8e-efe503064286 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:25.946963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b98008-79e7-407a-9a8e-efe503064286 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:25.947455 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b98008-79e7-407a-9a8e-efe503064286 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:25.948436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b98008-79e7-407a-9a8e-efe503064286 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:25.948951 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b98008-79e7-407a-9a8e-efe503064286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:25.950827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b98008-79e7-407a-9a8e-efe503064286 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:25.958575 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb266d5d-3519-4c24-9800-ecfcf124036b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:25.959397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb266d5d-3519-4c24-9800-ecfcf124036b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:25.959683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb266d5d-3519-4c24-9800-ecfcf124036b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:25.959961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb266d5d-3519-4c24-9800-ecfcf124036b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:25.961546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 432/857] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:25.968745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b98008-79e7-407a-9a8e-efe503064286 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:25.974758 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb266d5d-3519-4c24-9800-ecfcf124036b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:25.976236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb266d5d-3519-4c24-9800-ecfcf124036b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:25.994489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-bb266d5d-3519-4c24-9800-ecfcf124036b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:25.994489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:25.994489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:25.994489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:25.994489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:25.994489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:25.994489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:25.994489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:25.994489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:25.994489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:25.994489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:25.994489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:25.994489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:25.994489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:25.994489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:25.995443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:25.996566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:25.996566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:25.996566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:25.996566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:25.996566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:25.996566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:25.996566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:25.996566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:25.996566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 426/858] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:26.001283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb266d5d-3519-4c24-9800-ecfcf124036b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:26.019695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:26.024652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:26.107500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b98008-79e7-407a-9a8e-efe503064286 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:26.113292 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb266d5d-3519-4c24-9800-ecfcf124036b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:26.141197 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44156eb4-6c0b-483d-b1d1-326e9120a69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:26.142112 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44156eb4-6c0b-483d-b1d1-326e9120a69a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.142518 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44156eb4-6c0b-483d-b1d1-326e9120a69a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.142798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44156eb4-6c0b-483d-b1d1-326e9120a69a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.150910 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872359b0-5e3d-43f5-b8b9-4bc89aa1a17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:26.151573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872359b0-5e3d-43f5-b8b9-4bc89aa1a17b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.151859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872359b0-5e3d-43f5-b8b9-4bc89aa1a17b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.152136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872359b0-5e3d-43f5-b8b9-4bc89aa1a17b None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.152968 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872359b0-5e3d-43f5-b8b9-4bc89aa1a17b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:26.153277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872359b0-5e3d-43f5-b8b9-4bc89aa1a17b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:26.154366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44156eb4-6c0b-483d-b1d1-326e9120a69a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:26.154699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872359b0-5e3d-43f5-b8b9-4bc89aa1a17b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:26.155794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44156eb4-6c0b-483d-b1d1-326e9120a69a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:26.166957 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 427/859] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:26.167957 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36c239-1c2e-4562-ae41-8004640cc38d None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.168246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36c239-1c2e-4562-ae41-8004640cc38d None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.168567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36c239-1c2e-4562-ae41-8004640cc38d None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.169963 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 428/860] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:26.173251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-44156eb4-6c0b-483d-b1d1-326e9120a69a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:26.173251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:26.173251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:26.173251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:26.173251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:26.173251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:26.173251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:26.173251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:26.173251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:26.173251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:26.173251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:26.173251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:26.173251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:26.173251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:26.173251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:26.174087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:26.179786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:26.179786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:26.179786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:26.179786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:26.180004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36c239-1c2e-4562-ae41-8004640cc38d None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:26.180083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 433/861] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:26.180083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f19b05-ed22-4ee3-89d6-a28f997c2a21 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.180083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f19b05-ed22-4ee3-89d6-a28f997c2a21 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.180083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f19b05-ed22-4ee3-89d6-a28f997c2a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.195221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c4f19b05-ed22-4ee3-89d6-a28f997c2a21 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:26.196458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c4f19b05-ed22-4ee3-89d6-a28f997c2a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:26.263972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:26.272469 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 434/862] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:26.272469 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f19b05-ed22-4ee3-89d6-a28f997c2a21 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:26.292253 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:26.299597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36c239-1c2e-4562-ae41-8004640cc38d None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:26.339546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19128d54-92d4-4d1d-872c-c89f5bc0e6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:26.340235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19128d54-92d4-4d1d-872c-c89f5bc0e6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.340545 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19128d54-92d4-4d1d-872c-c89f5bc0e6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.340817 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19128d54-92d4-4d1d-872c-c89f5bc0e6a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.341655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19128d54-92d4-4d1d-872c-c89f5bc0e6a2 None admin] RBAC: Adding query filter params (role.id=7fe62c5c771d41c28f790fade1df9357, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=2d514236db644010859fc18250f5894e) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:26.341963 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19128d54-92d4-4d1d-872c-c89f5bc0e6a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:26.343416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19128d54-92d4-4d1d-872c-c89f5bc0e6a2 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:26.360680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 429/863] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:26 2022] GET /identity/v3/role_assignments?user.id=2d514236db644010859fc18250f5894e&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=7fe62c5c771d41c28f790fade1df9357 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:26.361459 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19128d54-92d4-4d1d-872c-c89f5bc0e6a2 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:26.397431 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f19b05-ed22-4ee3-89d6-a28f997c2a21 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:26.438886 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae999fba-cc95-4092-b5a3-8417e7175b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:26.439990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae999fba-cc95-4092-b5a3-8417e7175b0a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.440520 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae999fba-cc95-4092-b5a3-8417e7175b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.441002 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae999fba-cc95-4092-b5a3-8417e7175b0a None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.442508 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae999fba-cc95-4092-b5a3-8417e7175b0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:26.443050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae999fba-cc95-4092-b5a3-8417e7175b0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:26.444903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae999fba-cc95-4092-b5a3-8417e7175b0a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:26.459605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 435/864] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:26.468357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae999fba-cc95-4092-b5a3-8417e7175b0a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:26.487168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19128d54-92d4-4d1d-872c-c89f5bc0e6a2 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:26.537151 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d78c5d-5a4b-463c-8c5b-2e5a0035ff04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:26.537824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d78c5d-5a4b-463c-8c5b-2e5a0035ff04 None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.538143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d78c5d-5a4b-463c-8c5b-2e5a0035ff04 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.538378 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d78c5d-5a4b-463c-8c5b-2e5a0035ff04 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.539107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d78c5d-5a4b-463c-8c5b-2e5a0035ff04 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:26.540530 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d78c5d-5a4b-463c-8c5b-2e5a0035ff04 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:26.550908 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:26.560736 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 430/865] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jul 20 19:11:26 2022] POST /identity/v3/services => generated 269 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:26.574007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:26.604346 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae999fba-cc95-4092-b5a3-8417e7175b0a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:26.654061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4b992-b686-409a-994d-6de80804fd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:26.654951 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4b992-b686-409a-994d-6de80804fd4b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.655234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4b992-b686-409a-994d-6de80804fd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.655542 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4b992-b686-409a-994d-6de80804fd4b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.664564 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4b992-b686-409a-994d-6de80804fd4b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:26.666089 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4b992-b686-409a-994d-6de80804fd4b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:26.682190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-0fb4b992-b686-409a-994d-6de80804fd4b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:11:26.682190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:26.682190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:26.682190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:26.682190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:26.682190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:26.682190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:26.682190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:26.682190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:26.682190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:26.682190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:26.682190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:26.682190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:26.682190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:26.682190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:26.682190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:26.683642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:26.684854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:26.684854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:26.684854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:26.684854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:26.684854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:26.684854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:26.684854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:26.684854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:26.684854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:11:26.684854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:26.684854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 436/866] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:26.689890 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d78c5d-5a4b-463c-8c5b-2e5a0035ff04 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:26.796139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d78c5d-5a4b-463c-8c5b-2e5a0035ff04 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:26.843431 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7246b-f12c-458f-977e-860eac67a537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:26.844106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7246b-f12c-458f-977e-860eac67a537 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.844429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7246b-f12c-458f-977e-860eac67a537 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.844705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7246b-f12c-458f-977e-860eac67a537 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.845696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7246b-f12c-458f-977e-860eac67a537 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:26.846006 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7246b-f12c-458f-977e-860eac67a537 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:26.847432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7246b-f12c-458f-977e-860eac67a537 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:26.866342 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 431/867] 10.222.0.43 () {56 vars in 1135 bytes} [Wed Jul 20 19:11:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:26.877410 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4b992-b686-409a-994d-6de80804fd4b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:26.966021 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e7754-0f56-4ee7-a09a-60bffa04b3e4 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.966211 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e7754-0f56-4ee7-a09a-60bffa04b3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.966492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e7754-0f56-4ee7-a09a-60bffa04b3e4 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.968155 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 432/868] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:26.973938 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cbf55-eeb6-4781-9acf-a60a89b8b2d5 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.974356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cbf55-eeb6-4781-9acf-a60a89b8b2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.974731 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cbf55-eeb6-4781-9acf-a60a89b8b2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.984109 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4b992-b686-409a-994d-6de80804fd4b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:26.992985 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-5a8cbf55-eeb6-4781-9acf-a60a89b8b2d5 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:26.994416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5a8cbf55-eeb6-4781-9acf-a60a89b8b2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:27.015887 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:27.023129 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac962ab5-35f8-4ec0-8e58-7961c02527dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:27.023873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac962ab5-35f8-4ec0-8e58-7961c02527dc None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:27.024231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac962ab5-35f8-4ec0-8e58-7961c02527dc None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:27.024607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac962ab5-35f8-4ec0-8e58-7961c02527dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:27.058267 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:27.084433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 433/869] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:26 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:27.108653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8cbf55-eeb6-4781-9acf-a60a89b8b2d5 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:27.114346 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac962ab5-35f8-4ec0-8e58-7961c02527dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:27.115811 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac962ab5-35f8-4ec0-8e58-7961c02527dc None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:27.211792 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8cbf55-eeb6-4781-9acf-a60a89b8b2d5 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:27.226953 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 437/870] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:26 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:11:27.227882 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac962ab5-35f8-4ec0-8e58-7961c02527dc None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:27.250569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723611c3-e535-49a9-ad61-614c378af87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:27.251560 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723611c3-e535-49a9-ad61-614c378af87a None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:27.252039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723611c3-e535-49a9-ad61-614c378af87a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:27.252559 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723611c3-e535-49a9-ad61-614c378af87a None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:27.253617 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723611c3-e535-49a9-ad61-614c378af87a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:27.255301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723611c3-e535-49a9-ad61-614c378af87a None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:27.261537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:27.273212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 434/871] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jul 20 19:11:27 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:11:27.348929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac962ab5-35f8-4ec0-8e58-7961c02527dc None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:27.399958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48feed-809d-4d1a-a8dc-1bd929058020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:27.400791 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48feed-809d-4d1a-a8dc-1bd929058020 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:27.401075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48feed-809d-4d1a-a8dc-1bd929058020 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:27.401347 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48feed-809d-4d1a-a8dc-1bd929058020 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:27.413651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48feed-809d-4d1a-a8dc-1bd929058020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:27.415122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48feed-809d-4d1a-a8dc-1bd929058020 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:27.434392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-3f48feed-809d-4d1a-a8dc-1bd929058020 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:27.434392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:27.434392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:27.434392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:27.434392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:27.434392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:27.434392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:27.434392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:27.434392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:27.434392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:27.434392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:27.434392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:27.434392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:27.434392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:27.434392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:27.435342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:27.436236 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:27.436236 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:27.436236 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:27.436236 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:27.436236 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 438/872] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:27.442567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723611c3-e535-49a9-ad61-614c378af87a None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:27.494222 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:27.549618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723611c3-e535-49a9-ad61-614c378af87a None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:27.584004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1fbdd3-30bc-4af7-8047-7ce782a648bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:27.584710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fbdd3-30bc-4af7-8047-7ce782a648bc None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:27.585037 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fbdd3-30bc-4af7-8047-7ce782a648bc None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:27.585315 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fbdd3-30bc-4af7-8047-7ce782a648bc None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:27.586038 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1fbdd3-30bc-4af7-8047-7ce782a648bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:27.586342 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1fbdd3-30bc-4af7-8047-7ce782a648bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:27.587687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1fbdd3-30bc-4af7-8047-7ce782a648bc None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:27.600161 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 435/873] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:27.605242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48feed-809d-4d1a-a8dc-1bd929058020 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:27.695034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:27.715024 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48feed-809d-4d1a-a8dc-1bd929058020 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:27.750799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0ec8e-c19c-4ce4-add7-e4fb22d7e2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:27.751517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0ec8e-c19c-4ce4-add7-e4fb22d7e2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:27.751802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0ec8e-c19c-4ce4-add7-e4fb22d7e2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:27.752080 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0ec8e-c19c-4ce4-add7-e4fb22d7e2a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:27.759331 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0ec8e-c19c-4ce4-add7-e4fb22d7e2a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:27.760439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0ec8e-c19c-4ce4-add7-e4fb22d7e2a1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:27.774237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-2cf0ec8e-c19c-4ce4-add7-e4fb22d7e2a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:27.774237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:27.774237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:27.774237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:27.774237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:27.774237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:27.774237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:27.774237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:27.774237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:27.774237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:27.774237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:27.774237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:27.774237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:27.774237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:27.774237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:27.775126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:27.776249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:27.776249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:27.776249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:27.776249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:27.776249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:27.776249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:27.776249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:27.776249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:27.776249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 439/874] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:27.781263 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1fbdd3-30bc-4af7-8047-7ce782a648bc None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:27.888135 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1fbdd3-30bc-4af7-8047-7ce782a648bc None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:27.895723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:27.925451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb1b9f-6003-4998-8c95-d249c2c5ee3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:27.926657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb1b9f-6003-4998-8c95-d249c2c5ee3d None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:27.926951 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb1b9f-6003-4998-8c95-d249c2c5ee3d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:27.927228 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb1b9f-6003-4998-8c95-d249c2c5ee3d None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:27.928016 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb1b9f-6003-4998-8c95-d249c2c5ee3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:27.928346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb1b9f-6003-4998-8c95-d249c2c5ee3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:27.929766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb1b9f-6003-4998-8c95-d249c2c5ee3d None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:27.942277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 436/875] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:27.948330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0ec8e-c19c-4ce4-add7-e4fb22d7e2a1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:28.052699 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0ec8e-c19c-4ce4-add7-e4fb22d7e2a1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:28.093712 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10a400-fedd-451a-b408-debac98c6e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:28.094671 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10a400-fedd-451a-b408-debac98c6e03 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:28.095132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10a400-fedd-451a-b408-debac98c6e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:28.095559 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10a400-fedd-451a-b408-debac98c6e03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:28.096570 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10a400-fedd-451a-b408-debac98c6e03 None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=1c944eab8a8948749bb8375787216b09) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:28.097029 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10a400-fedd-451a-b408-debac98c6e03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:28.098639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10a400-fedd-451a-b408-debac98c6e03 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:28.106227 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 440/876] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:27 2022] GET /identity/v3/role_assignments?user.id=1c944eab8a8948749bb8375787216b09&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:28.615728 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e941-e423-4296-84fc-f46312086a40 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:28.615929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e941-e423-4296-84fc-f46312086a40 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:28.616212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e941-e423-4296-84fc-f46312086a40 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:28.617843 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 437/877] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:28.623377 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a88cf-bdd1-4dbb-8ec5-12b26385e217 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:28.623848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a88cf-bdd1-4dbb-8ec5-12b26385e217 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:28.623926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a88cf-bdd1-4dbb-8ec5-12b26385e217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:28.631666 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:28.643111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3f2a88cf-bdd1-4dbb-8ec5-12b26385e217 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:28.644461 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3f2a88cf-bdd1-4dbb-8ec5-12b26385e217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:28.715749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 441/878] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:28.739452 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77e941-e423-4296-84fc-f46312086a40 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:28.784234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75439ca0-c5ed-4bfc-bf90-f275e0d9404e None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:28.784563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75439ca0-c5ed-4bfc-bf90-f275e0d9404e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:28.784820 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75439ca0-c5ed-4bfc-bf90-f275e0d9404e None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:28.786487 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 442/879] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:28.791666 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0275d-605a-4f92-9319-4ef861d4f3a8 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:28.791948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0275d-605a-4f92-9319-4ef861d4f3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:28.792220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0275d-605a-4f92-9319-4ef861d4f3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:28.812174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a9f0275d-605a-4f92-9319-4ef861d4f3a8 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:28.813498 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a9f0275d-605a-4f92-9319-4ef861d4f3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:28.856596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77e941-e423-4296-84fc-f46312086a40 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:28.900478 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 443/880] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:28.906239 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f07da5-6119-4b86-a248-cd7461e0d98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:28.907125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f07da5-6119-4b86-a248-cd7461e0d98a None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:28.907406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f07da5-6119-4b86-a248-cd7461e0d98a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:28.907721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f07da5-6119-4b86-a248-cd7461e0d98a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:28.920345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f07da5-6119-4b86-a248-cd7461e0d98a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:28.924493 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0275d-605a-4f92-9319-4ef861d4f3a8 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:28.929726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-32f07da5-6119-4b86-a248-cd7461e0d98a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:28.929726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:28.929726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:28.929726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:28.929726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:28.929726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:28.929726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:28.929726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:28.929726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:28.929726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:28.929726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:28.929726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:28.929726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:28.929726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:28.929726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:28.929726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:28.930838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:28.930838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:28.930838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:28.930838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:28.930838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:28.930838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:28.930838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:28.930838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:28.930838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:28.930838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:28.930838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:28.930838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:28.930838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:28.930838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:28.930838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:28.930838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:28.931725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:28.931725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:28.931725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:28.931725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:28.931725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:28.931725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:28.931725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:28.931725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:28.931725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:28.931725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:28.931725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:28.931725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:28.931725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 438/881] 10.222.0.43 () {56 vars in 1120 bytes} [Wed Jul 20 19:11:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:28.937188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f07da5-6119-4b86-a248-cd7461e0d98a None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:28.969609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:28.991975 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:29.032208 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0275d-605a-4f92-9319-4ef861d4f3a8 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.070105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f07da5-6119-4b86-a248-cd7461e0d98a None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.071329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a0098-4921-42d8-ad26-a82ac000750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.072586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a0098-4921-42d8-ad26-a82ac000750e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.072934 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a0098-4921-42d8-ad26-a82ac000750e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.073286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a0098-4921-42d8-ad26-a82ac000750e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.074026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a0098-4921-42d8-ad26-a82ac000750e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.075493 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a0098-4921-42d8-ad26-a82ac000750e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.089434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-7d8a0098-4921-42d8-ad26-a82ac000750e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:11:29.089434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:29.089434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:29.089434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:29.089434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:29.089434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:29.089434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:29.089434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:29.089434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:11:29.089434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:29.089434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:29.089434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:29.089434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:29.089434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:29.089434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:29.089434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:29.090388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:29.090388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:29.090388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:29.090388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:29.090388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:29.090388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:29.090388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:29.090388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:29.090388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:29.090388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:29.090388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:29.090388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:29.090388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:29.090388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:29.090388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:29.090388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:29.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:29.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:29.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:29.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:29.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:29.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:29.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:29.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:29.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:29.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:29.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:29.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:29.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:29.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:11:29.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:29.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 444/882] 10.222.0.43 () {56 vars in 1132 bytes} [Wed Jul 20 19:11:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:29.096959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a0098-4921-42d8-ad26-a82ac000750e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.120928 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af95bf3-5f8b-4370-a5c3-a0f61a9d8b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.121728 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af95bf3-5f8b-4370-a5c3-a0f61a9d8b99 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.122127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af95bf3-5f8b-4370-a5c3-a0f61a9d8b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.122946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af95bf3-5f8b-4370-a5c3-a0f61a9d8b99 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.123989 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af95bf3-5f8b-4370-a5c3-a0f61a9d8b99 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:29.124324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af95bf3-5f8b-4370-a5c3-a0f61a9d8b99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.126412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af95bf3-5f8b-4370-a5c3-a0f61a9d8b99 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.143961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 439/883] 10.222.0.43 () {56 vars in 1115 bytes} [Wed Jul 20 19:11:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:29.145681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af95bf3-5f8b-4370-a5c3-a0f61a9d8b99 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.220026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a0098-4921-42d8-ad26-a82ac000750e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.250422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:29.260297 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ff993-7e24-47ad-a360-eec49f7fe240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.260989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ff993-7e24-47ad-a360-eec49f7fe240 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.261408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ff993-7e24-47ad-a360-eec49f7fe240 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.261691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ff993-7e24-47ad-a360-eec49f7fe240 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.262407 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ff993-7e24-47ad-a360-eec49f7fe240 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:29.262786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ff993-7e24-47ad-a360-eec49f7fe240 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.264265 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ff993-7e24-47ad-a360-eec49f7fe240 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.268136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:29.272711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 445/884] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:29.275412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ff993-7e24-47ad-a360-eec49f7fe240 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.287606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af95bf3-5f8b-4370-a5c3-a0f61a9d8b99 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.338121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d27206-66ad-4b72-b14f-c168370b63c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.338929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d27206-66ad-4b72-b14f-c168370b63c1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.339217 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d27206-66ad-4b72-b14f-c168370b63c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.339493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d27206-66ad-4b72-b14f-c168370b63c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.362334 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d27206-66ad-4b72-b14f-c168370b63c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.364248 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d27206-66ad-4b72-b14f-c168370b63c1 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.383477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-16d27206-66ad-4b72-b14f-c168370b63c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:29.383477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:29.383477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:29.383477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:29.383477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:29.383477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:29.383477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:29.383477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:29.383477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:29.383477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:29.383477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:29.383477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:29.383477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:29.383477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:29.383477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:29.384455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:29.385356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:29.385356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:29.385356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:29.385356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:29.392490 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 440/885] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:29.393842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d27206-66ad-4b72-b14f-c168370b63c1 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.396772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ff993-7e24-47ad-a360-eec49f7fe240 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.444343 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3dc31-7123-48fc-a565-ea30f446a01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.445057 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3dc31-7123-48fc-a565-ea30f446a01f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.445469 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3dc31-7123-48fc-a565-ea30f446a01f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.445793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3dc31-7123-48fc-a565-ea30f446a01f None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.446904 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3dc31-7123-48fc-a565-ea30f446a01f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:29.447223 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3dc31-7123-48fc-a565-ea30f446a01f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.449073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3dc31-7123-48fc-a565-ea30f446a01f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.455537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 446/886] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:29.463059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3dc31-7123-48fc-a565-ea30f446a01f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.510293 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d27206-66ad-4b72-b14f-c168370b63c1 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.525338 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:29.548073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159254ef-1bf5-46c4-b220-3a74fda4e33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.548756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159254ef-1bf5-46c4-b220-3a74fda4e33b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.549037 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159254ef-1bf5-46c4-b220-3a74fda4e33b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.549310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159254ef-1bf5-46c4-b220-3a74fda4e33b None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.550039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159254ef-1bf5-46c4-b220-3a74fda4e33b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:29.550349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159254ef-1bf5-46c4-b220-3a74fda4e33b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.552243 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159254ef-1bf5-46c4-b220-3a74fda4e33b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.563027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:29.566754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 441/887] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:29.572774 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da3e91-6b75-4f79-affa-faeed50665fd None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.573086 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da3e91-6b75-4f79-affa-faeed50665fd None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.573347 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da3e91-6b75-4f79-affa-faeed50665fd None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.573793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3dc31-7123-48fc-a565-ea30f446a01f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.576326 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 442/888] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:29.576326 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da3e91-6b75-4f79-affa-faeed50665fd None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.614024 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a96f1-b31e-40c6-90fa-e7ef0c44b551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.614962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a96f1-b31e-40c6-90fa-e7ef0c44b551 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.615240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a96f1-b31e-40c6-90fa-e7ef0c44b551 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.615515 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a96f1-b31e-40c6-90fa-e7ef0c44b551 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.616313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a96f1-b31e-40c6-90fa-e7ef0c44b551 None admin] RBAC: Adding query filter params (interface=public, service_id=c8677f535092444ca68b4ffea9af4e35, region_id=RegionOne) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:29.616643 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a96f1-b31e-40c6-90fa-e7ef0c44b551 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.618267 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a96f1-b31e-40c6-90fa-e7ef0c44b551 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.623557 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 447/889] 10.222.0.43 () {56 vars in 1251 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/endpoints?service_id=c8677f535092444ca68b4ffea9af4e35&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:29.624621 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3d1c0-3f21-4595-8b48-36d275d0d151 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.624905 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3d1c0-3f21-4595-8b48-36d275d0d151 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.625179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3d1c0-3f21-4595-8b48-36d275d0d151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.641508 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-65b3d1c0-3f21-4595-8b48-36d275d0d151 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:29.642676 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-65b3d1c0-3f21-4595-8b48-36d275d0d151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:29.688362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da3e91-6b75-4f79-affa-faeed50665fd None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.713966 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 448/890] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:29.714908 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3d1c0-3f21-4595-8b48-36d275d0d151 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.734898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03a88b-7287-42c0-b8f5-4c24d4136ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.735836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03a88b-7287-42c0-b8f5-4c24d4136ded None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.736192 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03a88b-7287-42c0-b8f5-4c24d4136ded None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.736561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03a88b-7287-42c0-b8f5-4c24d4136ded None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.745224 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03a88b-7287-42c0-b8f5-4c24d4136ded None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.746901 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03a88b-7287-42c0-b8f5-4c24d4136ded None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.763547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-ce03a88b-7287-42c0-b8f5-4c24d4136ded None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:11:29.763547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:29.763547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:29.763547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:29.763547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:29.763547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:29.763547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:29.763547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:29.763547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:29.763547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:29.763547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:29.763547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:29.763547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:29.763547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:29.763547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:29.763547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:29.764523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:29.765471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:29.765471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:29.765471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:29.765471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:29.765471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:29.765471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:29.765471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:29.765471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:29.765471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:11:29.765471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:29.765471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 443/891] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:29.766049 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03a88b-7287-42c0-b8f5-4c24d4136ded None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.780348 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:29.833112 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3d1c0-3f21-4595-8b48-36d275d0d151 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.838528 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:29.872754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05e0ca-0873-43c5-b4c4-1c94887ec388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.873661 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05e0ca-0873-43c5-b4c4-1c94887ec388 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.874151 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05e0ca-0873-43c5-b4c4-1c94887ec388 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.874327 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05e0ca-0873-43c5-b4c4-1c94887ec388 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.875004 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05e0ca-0873-43c5-b4c4-1c94887ec388 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.876504 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05e0ca-0873-43c5-b4c4-1c94887ec388 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.880138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03a88b-7287-42c0-b8f5-4c24d4136ded None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.892652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 449/892] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:11:29.892652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05e0ca-0873-43c5-b4c4-1c94887ec388 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.931148 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd3c07-f1af-4043-b0ad-430e2b35391b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.931999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd3c07-f1af-4043-b0ad-430e2b35391b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.932298 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd3c07-f1af-4043-b0ad-430e2b35391b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.933148 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd3c07-f1af-4043-b0ad-430e2b35391b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.934453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd3c07-f1af-4043-b0ad-430e2b35391b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.936135 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd3c07-f1af-4043-b0ad-430e2b35391b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-6cbd3c07-f1af-4043-b0ad-430e2b35391b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:29.953943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:29.955047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:29.955047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:29.955047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:29.955047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:29.955047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:29.955047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:29.955047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:29.955047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:29.955047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:29.955047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:29.955047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:29.955047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:29.955047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:29.955047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:29.955047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:29.955047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:29.955852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:29.955852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:29.955852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:29.955852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:29.955852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:29.955852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:29.955852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:29.955852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:29.955852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:29.955852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:29.955852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:29.955852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:29.955852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:11:29.955852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:29.955852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 444/893] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:29.966206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd3c07-f1af-4043-b0ad-430e2b35391b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:30.023713 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05e0ca-0873-43c5-b4c4-1c94887ec388 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:30.043181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:30.077137 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e8244-af88-4662-891a-8a2160c9d5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:30.077958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e8244-af88-4662-891a-8a2160c9d5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:30.078461 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e8244-af88-4662-891a-8a2160c9d5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:30.078558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e8244-af88-4662-891a-8a2160c9d5c2 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:30.079801 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e8244-af88-4662-891a-8a2160c9d5c2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:30.080107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e8244-af88-4662-891a-8a2160c9d5c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:30.082867 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e8244-af88-4662-891a-8a2160c9d5c2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:30.103493 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 450/894] 10.222.0.43 () {56 vars in 1149 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:30.104324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd3c07-f1af-4043-b0ad-430e2b35391b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:30.111117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e8244-af88-4662-891a-8a2160c9d5c2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:30.133220 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:30.146241 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2e5be-26b4-4807-9e4d-596daf9fc19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:30.147207 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2e5be-26b4-4807-9e4d-596daf9fc19f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:30.147633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2e5be-26b4-4807-9e4d-596daf9fc19f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:30.148056 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2e5be-26b4-4807-9e4d-596daf9fc19f None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:30.148967 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2e5be-26b4-4807-9e4d-596daf9fc19f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:30.149424 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2e5be-26b4-4807-9e4d-596daf9fc19f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:30.150984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2e5be-26b4-4807-9e4d-596daf9fc19f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:30.156246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 445/895] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:30.165313 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2e5be-26b4-4807-9e4d-596daf9fc19f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:30.241811 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e8244-af88-4662-891a-8a2160c9d5c2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:30.282047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bc604-c1c1-41cf-a495-398b46c69f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:30.282701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bc604-c1c1-41cf-a495-398b46c69f18 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:30.283049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bc604-c1c1-41cf-a495-398b46c69f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:30.283460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bc604-c1c1-41cf-a495-398b46c69f18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:30.289142 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2e5be-26b4-4807-9e4d-596daf9fc19f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:30.324488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95088bc-b896-4fb9-a123-cd58f0d49f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:30.325162 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95088bc-b896-4fb9-a123-cd58f0d49f3b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:30.325449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95088bc-b896-4fb9-a123-cd58f0d49f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:30.325727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95088bc-b896-4fb9-a123-cd58f0d49f3b None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:30.326487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95088bc-b896-4fb9-a123-cd58f0d49f3b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:30.326804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95088bc-b896-4fb9-a123-cd58f0d49f3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:30.328159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95088bc-b896-4fb9-a123-cd58f0d49f3b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:30.329623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:30.338499 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 446/896] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:11:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:30.339965 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb04990-4586-4a9d-b66d-76c055f473c3 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:30.340267 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb04990-4586-4a9d-b66d-76c055f473c3 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:30.340574 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb04990-4586-4a9d-b66d-76c055f473c3 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:30.342361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 447/897] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:30.343076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb04990-4586-4a9d-b66d-76c055f473c3 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:30.367391 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7bc604-c1c1-41cf-a495-398b46c69f18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:30.368895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7bc604-c1c1-41cf-a495-398b46c69f18 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:30.407933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:30.460754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb04990-4586-4a9d-b66d-76c055f473c3 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:30.481169 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 451/898] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:30 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:11:30.482356 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf9399-db6d-4de9-9644-74952669d674 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:30.482646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf9399-db6d-4de9-9644-74952669d674 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:30.482926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf9399-db6d-4de9-9644-74952669d674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:30.500638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f4cf9399-db6d-4de9-9644-74952669d674 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:30.500979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8c317-e03e-44ba-b88d-3abd9017fe60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:30.501641 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8c317-e03e-44ba-b88d-3abd9017fe60 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:30.501835 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f4cf9399-db6d-4de9-9644-74952669d674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:30.501958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8c317-e03e-44ba-b88d-3abd9017fe60 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:30.502179 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8c317-e03e-44ba-b88d-3abd9017fe60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:30.502975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8c317-e03e-44ba-b88d-3abd9017fe60 None admin] RBAC: Adding query filter params (interface=public, service_id=fe53eadd8e434836bb92f0a2310ede74, region_id=RegionOne) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:30.503277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8c317-e03e-44ba-b88d-3abd9017fe60 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:30.504791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8c317-e03e-44ba-b88d-3abd9017fe60 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:30.512766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 448/899] 10.222.0.43 () {56 vars in 1251 bytes} [Wed Jul 20 19:11:30 2022] GET /identity/v3/endpoints?service_id=fe53eadd8e434836bb92f0a2310ede74&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:30.513560 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8c317-e03e-44ba-b88d-3abd9017fe60 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:30.582349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 452/900] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:30.583196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf9399-db6d-4de9-9644-74952669d674 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:30.585652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:30.636431 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8c317-e03e-44ba-b88d-3abd9017fe60 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:30.654336 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:30.674910 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc1c80-a431-4de8-8459-0a829162c95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:30.675710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc1c80-a431-4de8-8459-0a829162c95b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:30.676058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc1c80-a431-4de8-8459-0a829162c95b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:30.676845 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc1c80-a431-4de8-8459-0a829162c95b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:30.688199 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc1c80-a431-4de8-8459-0a829162c95b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:30.689661 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc1c80-a431-4de8-8459-0a829162c95b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:30.691325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf9399-db6d-4de9-9644-74952669d674 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:30.707683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-6fcc1c80-a431-4de8-8459-0a829162c95b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:30.707683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:30.707683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:30.707683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:30.707683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:30.707683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:30.707683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:30.707683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:30.707683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:30.707683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:30.707683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:30.707683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:30.707683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:30.707683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:30.707683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:30.708652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:30.709551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:30.709551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:30.709551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:30.709551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:30.709551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 449/901] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:30.710389 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc1c80-a431-4de8-8459-0a829162c95b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:30.734767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47246d04-d503-4e02-95a3-602c929bb50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:30.735706 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47246d04-d503-4e02-95a3-602c929bb50d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:30.736203 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47246d04-d503-4e02-95a3-602c929bb50d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:30.736533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47246d04-d503-4e02-95a3-602c929bb50d None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:30.737226 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47246d04-d503-4e02-95a3-602c929bb50d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:30.739137 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47246d04-d503-4e02-95a3-602c929bb50d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:30.752327 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 453/902] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jul 20 19:11:30 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:11:30.752327 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47246d04-d503-4e02-95a3-602c929bb50d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:30.827683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc1c80-a431-4de8-8459-0a829162c95b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:30.836863 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:30.869128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565e672-b7a0-4657-8ddd-bbe9dfd55404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:30.870417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565e672-b7a0-4657-8ddd-bbe9dfd55404 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:30.870899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565e672-b7a0-4657-8ddd-bbe9dfd55404 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:30.871449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565e672-b7a0-4657-8ddd-bbe9dfd55404 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:30.872447 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565e672-b7a0-4657-8ddd-bbe9dfd55404 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:30.874139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565e672-b7a0-4657-8ddd-bbe9dfd55404 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:30.881156 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47246d04-d503-4e02-95a3-602c929bb50d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-0565e672-b7a0-4657-8ddd-bbe9dfd55404 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:30.888143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:30.889600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:30.889600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:30.889600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:30.889600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:30.889600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:30.889600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:30.889600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:30.889600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:30.889600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:30.889600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:30.889600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:30.889600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:30.889600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:30.889600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:30.889600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:30.889600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:30.890435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:30.890435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:30.890435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:30.890435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:30.890435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:30.890435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:30.890435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:30.890435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:30.890435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:30.890435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:30.890435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:30.890435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:30.890435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:30.890435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:30.890435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 450/903] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:30.895878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565e672-b7a0-4657-8ddd-bbe9dfd55404 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:30.933239 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:30.934641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd9869-cfe9-41e0-bdca-0c324292c7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:30.936255 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd9869-cfe9-41e0-bdca-0c324292c7fb None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:30.936255 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd9869-cfe9-41e0-bdca-0c324292c7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:30.936255 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd9869-cfe9-41e0-bdca-0c324292c7fb None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:30.937032 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd9869-cfe9-41e0-bdca-0c324292c7fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:30.937518 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd9869-cfe9-41e0-bdca-0c324292c7fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:30.939683 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd9869-cfe9-41e0-bdca-0c324292c7fb None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:30.956485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 454/904] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:30.964517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd9869-cfe9-41e0-bdca-0c324292c7fb None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:31.023180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565e672-b7a0-4657-8ddd-bbe9dfd55404 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:31.067132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6243b82-2d7d-4f01-86b7-8fe8531c2238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:31.067579 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6243b82-2d7d-4f01-86b7-8fe8531c2238 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:31.067860 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6243b82-2d7d-4f01-86b7-8fe8531c2238 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:31.068136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6243b82-2d7d-4f01-86b7-8fe8531c2238 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:31.068899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6243b82-2d7d-4f01-86b7-8fe8531c2238 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:31.069205 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6243b82-2d7d-4f01-86b7-8fe8531c2238 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:31.070563 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6243b82-2d7d-4f01-86b7-8fe8531c2238 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:31.080495 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 451/905] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:31.082911 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6243b82-2d7d-4f01-86b7-8fe8531c2238 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:31.088198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd9869-cfe9-41e0-bdca-0c324292c7fb None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:31.124444 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:31.137391 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457dd779-0428-42d1-8147-131be9a28c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:31.138693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457dd779-0428-42d1-8147-131be9a28c88 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:31.139127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457dd779-0428-42d1-8147-131be9a28c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:31.140002 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457dd779-0428-42d1-8147-131be9a28c88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:31.150338 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457dd779-0428-42d1-8147-131be9a28c88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:31.152493 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457dd779-0428-42d1-8147-131be9a28c88 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:31.172550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-457dd779-0428-42d1-8147-131be9a28c88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:31.172550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:31.172550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:31.172550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:31.172550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:31.172550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:31.172550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:31.172550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:31.172550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:31.172550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:31.172550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:31.172550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:31.172550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:31.172550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:31.172550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:31.173690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:31.174752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:31.174752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:31.174752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:31.174752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:31.174752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:31.174752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:31.174752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:31.174752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:31.182013 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 455/906] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:31.183856 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457dd779-0428-42d1-8147-131be9a28c88 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:31.197734 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:31.205679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6243b82-2d7d-4f01-86b7-8fe8531c2238 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:31.242660 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a15ac-8be9-41bf-988f-976383186931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:31.243353 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a15ac-8be9-41bf-988f-976383186931 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:31.243638 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a15ac-8be9-41bf-988f-976383186931 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:31.243906 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a15ac-8be9-41bf-988f-976383186931 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:31.244654 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a15ac-8be9-41bf-988f-976383186931 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:31.244954 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a15ac-8be9-41bf-988f-976383186931 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:31.246302 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a15ac-8be9-41bf-988f-976383186931 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:31.256604 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 452/907] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:31 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:31.314232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457dd779-0428-42d1-8147-131be9a28c88 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:31.355019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68976d98-55e8-4a12-9349-d2450af9e845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:31.355669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68976d98-55e8-4a12-9349-d2450af9e845 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:31.355952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68976d98-55e8-4a12-9349-d2450af9e845 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:31.356229 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68976d98-55e8-4a12-9349-d2450af9e845 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:31.357068 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68976d98-55e8-4a12-9349-d2450af9e845 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:31.357381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68976d98-55e8-4a12-9349-d2450af9e845 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:31.358822 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68976d98-55e8-4a12-9349-d2450af9e845 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:31.372275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 456/908] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:31.378331 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a15ac-8be9-41bf-988f-976383186931 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:31.487214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a15ac-8be9-41bf-988f-976383186931 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:31.527185 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840e4fe-9af9-4937-a327-37a2b19a4b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:31.528423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840e4fe-9af9-4937-a327-37a2b19a4b43 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:31.528933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840e4fe-9af9-4937-a327-37a2b19a4b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:31.529407 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840e4fe-9af9-4937-a327-37a2b19a4b43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:31.530472 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840e4fe-9af9-4937-a327-37a2b19a4b43 None admin] RBAC: Adding query filter params (role.id=35471af587764a54ba1c01a3ab111054, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=2d514236db644010859fc18250f5894e) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:31.530990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840e4fe-9af9-4937-a327-37a2b19a4b43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:31.532690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840e4fe-9af9-4937-a327-37a2b19a4b43 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:31.539798 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 453/909] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:31 2022] GET /identity/v3/role_assignments?user.id=2d514236db644010859fc18250f5894e&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=35471af587764a54ba1c01a3ab111054 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:32.006885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad3619-4715-4f2f-b169-448d44420f71 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:32.007438 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad3619-4715-4f2f-b169-448d44420f71 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:32.007438 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad3619-4715-4f2f-b169-448d44420f71 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:32.009071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 457/910] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:32.015150 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14add05d-dd3a-4a81-a130-a5b8f3212a34 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:32.015605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14add05d-dd3a-4a81-a130-a5b8f3212a34 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:32.016028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14add05d-dd3a-4a81-a130-a5b8f3212a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:32.034004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-14add05d-dd3a-4a81-a130-a5b8f3212a34 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:32.035222 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-14add05d-dd3a-4a81-a130-a5b8f3212a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:32.057869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:32.115144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 454/911] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:32.141857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad3619-4715-4f2f-b169-448d44420f71 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:32.208662 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:32.298785 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad3619-4715-4f2f-b169-448d44420f71 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:32.344314 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed1218-55ca-419b-aac4-fba76cd4489e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:32.344314 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed1218-55ca-419b-aac4-fba76cd4489e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:32.344314 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed1218-55ca-419b-aac4-fba76cd4489e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:32.344314 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed1218-55ca-419b-aac4-fba76cd4489e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:32.344736 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed1218-55ca-419b-aac4-fba76cd4489e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:32.344736 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed1218-55ca-419b-aac4-fba76cd4489e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:32.356017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-8fed1218-55ca-419b-aac4-fba76cd4489e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:11:32.356017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:32.356017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:32.356017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:32.356017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:32.356017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:32.356017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:32.356017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:32.356017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:11:32.356017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:32.356017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:32.356017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:32.356017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:32.356017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:32.356017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:32.356017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:32.357039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:32.357039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:32.357039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:32.357039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:32.357039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:32.357039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:32.357039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:32.357039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:32.357039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:32.357039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:32.357039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:32.357039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:32.357039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:32.357039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:32.357039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:32.357039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:32.357899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:32.357899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:32.357899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:32.357899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:32.357899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:32.357899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:32.357899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:32.357899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:32.357899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:32.357899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:32.357899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:32.357899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:32.357899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:32.357899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:11:32.357899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:32.357899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 458/912] 10.222.0.43 () {56 vars in 1132 bytes} [Wed Jul 20 19:11:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:32.363284 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14add05d-dd3a-4a81-a130-a5b8f3212a34 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:32.480449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14add05d-dd3a-4a81-a130-a5b8f3212a34 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:32.518368 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c43c6-073f-4036-b4ff-ef7143c2ad74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:32.519452 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c43c6-073f-4036-b4ff-ef7143c2ad74 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:32.520505 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c43c6-073f-4036-b4ff-ef7143c2ad74 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:32.520995 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c43c6-073f-4036-b4ff-ef7143c2ad74 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:32.521971 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c43c6-073f-4036-b4ff-ef7143c2ad74 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:32.522537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c43c6-073f-4036-b4ff-ef7143c2ad74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:32.524338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c43c6-073f-4036-b4ff-ef7143c2ad74 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:32.530760 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 455/913] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:32.536679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed1218-55ca-419b-aac4-fba76cd4489e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:32.650964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed1218-55ca-419b-aac4-fba76cd4489e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:32.664555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:32.691738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f53c24-484c-45c7-9593-09ed8bef0b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:32.692443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f53c24-484c-45c7-9593-09ed8bef0b35 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:32.692729 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f53c24-484c-45c7-9593-09ed8bef0b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:32.693008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f53c24-484c-45c7-9593-09ed8bef0b35 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:32.693731 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f53c24-484c-45c7-9593-09ed8bef0b35 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:32.694040 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f53c24-484c-45c7-9593-09ed8bef0b35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:32.695421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f53c24-484c-45c7-9593-09ed8bef0b35 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:32.702346 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 459/914] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:32.708278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c43c6-073f-4036-b4ff-ef7143c2ad74 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:32.738125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:32.838708 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c43c6-073f-4036-b4ff-ef7143c2ad74 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:32.882434 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b61a3-1b33-4540-bd16-1a377ef1a59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:32.883111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b61a3-1b33-4540-bd16-1a377ef1a59c None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:32.883398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b61a3-1b33-4540-bd16-1a377ef1a59c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:32.883677 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b61a3-1b33-4540-bd16-1a377ef1a59c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:32.884529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b61a3-1b33-4540-bd16-1a377ef1a59c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:32.885911 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b61a3-1b33-4540-bd16-1a377ef1a59c None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:32.914044 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 456/915] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jul 20 19:11:32 2022] POST /identity/v3/endpoints => generated 347 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:32.930514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed1349-564a-4d6b-8ff9-0bb80066d949 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:32.930814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed1349-564a-4d6b-8ff9-0bb80066d949 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:32.931100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed1349-564a-4d6b-8ff9-0bb80066d949 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:32.934575 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 460/916] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:32.939502 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da73d72-c09d-401c-a602-be3415449029 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:32.939803 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da73d72-c09d-401c-a602-be3415449029 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:32.940093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da73d72-c09d-401c-a602-be3415449029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:32.961079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-1da73d72-c09d-401c-a602-be3415449029 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:32.962562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-1da73d72-c09d-401c-a602-be3415449029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:33.043577 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:33.051040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 457/917] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:33.075416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed1349-564a-4d6b-8ff9-0bb80066d949 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:33.218895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed1349-564a-4d6b-8ff9-0bb80066d949 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:33.264626 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e91d6-b915-4365-bdc3-4bee0584b6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:33.265806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e91d6-b915-4365-bdc3-4bee0584b6ba None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:33.266274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e91d6-b915-4365-bdc3-4bee0584b6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:33.266730 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e91d6-b915-4365-bdc3-4bee0584b6ba None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:33.267575 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e91d6-b915-4365-bdc3-4bee0584b6ba None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:33.269222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e91d6-b915-4365-bdc3-4bee0584b6ba None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-ec5e91d6-b915-4365-bdc3-4bee0584b6ba None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:33.283920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:33.285083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:33.285083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:33.285083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:33.285083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:33.285083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:33.285083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:33.285083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:33.285083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:33.285083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:33.285083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:33.285083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:33.285083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:33.285083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:33.285083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:33.285083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:33.285083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:33.286284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:33.286284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:33.286284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:33.286284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:33.286284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:33.286284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:33.286284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:33.286284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:33.286284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:33.286284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:33.286284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:33.286284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:33.286284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:11:33.286284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:33.290410 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 461/918] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:11:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:33.291819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da73d72-c09d-401c-a602-be3415449029 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:33.409833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da73d72-c09d-401c-a602-be3415449029 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:33.445236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f2e77-c4e9-4523-8ca1-f82c006007e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:33.445906 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f2e77-c4e9-4523-8ca1-f82c006007e3 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:33.446184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f2e77-c4e9-4523-8ca1-f82c006007e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:33.446454 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f2e77-c4e9-4523-8ca1-f82c006007e3 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:33.447169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f2e77-c4e9-4523-8ca1-f82c006007e3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:33.447472 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f2e77-c4e9-4523-8ca1-f82c006007e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:33.448851 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f2e77-c4e9-4523-8ca1-f82c006007e3 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:33.458175 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 458/919] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:11:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:33.459555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e91d6-b915-4365-bdc3-4bee0584b6ba None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:33.502529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:33.564849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e91d6-b915-4365-bdc3-4bee0584b6ba None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:33.599691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f748d8a-4845-4c75-b7d1-b50facd93684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:33.600374 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f748d8a-4845-4c75-b7d1-b50facd93684 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:33.600658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f748d8a-4845-4c75-b7d1-b50facd93684 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:33.600930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f748d8a-4845-4c75-b7d1-b50facd93684 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:33.601633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f748d8a-4845-4c75-b7d1-b50facd93684 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:33.601939 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f748d8a-4845-4c75-b7d1-b50facd93684 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:33.603299 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f748d8a-4845-4c75-b7d1-b50facd93684 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:33.611316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 462/920] 10.222.0.43 () {56 vars in 1114 bytes} [Wed Jul 20 19:11:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:33.614485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff86fe2-5c15-43e2-8a58-6633a446ff39 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:33.614776 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff86fe2-5c15-43e2-8a58-6633a446ff39 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:33.615054 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff86fe2-5c15-43e2-8a58-6633a446ff39 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:33.616673 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 459/921] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:33.617451 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f748d8a-4845-4c75-b7d1-b50facd93684 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:33.622144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398a2a2-ffcd-40fc-9179-14762cb8ea81 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:33.622144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398a2a2-ffcd-40fc-9179-14762cb8ea81 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:33.622365 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398a2a2-ffcd-40fc-9179-14762cb8ea81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:33.639512 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-c398a2a2-ffcd-40fc-9179-14762cb8ea81 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:33.640980 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c398a2a2-ffcd-40fc-9179-14762cb8ea81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:33.723612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f748d8a-4845-4c75-b7d1-b50facd93684 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:33.725641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:33.729468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 460/922] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:33.730043 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:33.757331 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398a2a2-ffcd-40fc-9179-14762cb8ea81 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:33.769780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a79774-3646-4200-ad97-334abbc1146b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:33.770450 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a79774-3646-4200-ad97-334abbc1146b None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:33.770739 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a79774-3646-4200-ad97-334abbc1146b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:33.771013 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a79774-3646-4200-ad97-334abbc1146b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:33.772094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a79774-3646-4200-ad97-334abbc1146b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:33.773807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a79774-3646-4200-ad97-334abbc1146b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:33.808317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 463/923] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jul 20 19:11:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:33.829889 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002926b6-df3a-4c7d-afea-172eec943535 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:33.830229 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002926b6-df3a-4c7d-afea-172eec943535 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:33.830442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002926b6-df3a-4c7d-afea-172eec943535 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:33.835228 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 464/924] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:33.838768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57108bc8-2696-4f1d-bc2a-9ea54639a8ba None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:33.839370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57108bc8-2696-4f1d-bc2a-9ea54639a8ba None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:33.839944 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57108bc8-2696-4f1d-bc2a-9ea54639a8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:33.863010 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-57108bc8-2696-4f1d-bc2a-9ea54639a8ba None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:33.866310 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-57108bc8-2696-4f1d-bc2a-9ea54639a8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:33.885049 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398a2a2-ffcd-40fc-9179-14762cb8ea81 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:33.930068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a728e3d-cfa3-41e4-8ce5-01f41b948fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:33.931169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a728e3d-cfa3-41e4-8ce5-01f41b948fc1 None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:33.931672 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a728e3d-cfa3-41e4-8ce5-01f41b948fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:33.932165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a728e3d-cfa3-41e4-8ce5-01f41b948fc1 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:33.933439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a728e3d-cfa3-41e4-8ce5-01f41b948fc1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:33.935066 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a728e3d-cfa3-41e4-8ce5-01f41b948fc1 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:33.962580 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 461/925] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jul 20 19:11:33 2022] POST /identity/v3/services => generated 241 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:11:33.963401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 465/926] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:33.986682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a728e3d-cfa3-41e4-8ce5-01f41b948fc1 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:34.051693 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:34.124833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a728e3d-cfa3-41e4-8ce5-01f41b948fc1 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:34.166457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e99aa-add1-4c9c-be21-8e35c5676c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:34.167320 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e99aa-add1-4c9c-be21-8e35c5676c1e None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:34.167610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e99aa-add1-4c9c-be21-8e35c5676c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:34.167889 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e99aa-add1-4c9c-be21-8e35c5676c1e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:34.181148 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e99aa-add1-4c9c-be21-8e35c5676c1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:34.189778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-c79e99aa-add1-4c9c-be21-8e35c5676c1e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:34.189778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:34.189778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:34.189778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:34.189778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:34.189778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:34.189778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:34.189778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:34.189778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:34.189778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:34.189778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:34.189778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:34.189778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:34.189778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:34.189778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:34.189778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:34.190694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:34.190694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:34.190694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:34.190694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:34.190694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:34.190694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:34.190694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:34.190694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:34.190694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:34.190694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:34.190694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:34.190694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:34.190694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:34.190694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:34.190694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:34.190694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:34.191597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:34.191597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:34.191597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:34.191597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:34.191597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:34.191597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:34.191597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:34.191597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:34.191597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:34.191597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:34.191597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:34.191597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:34.191597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 462/927] 10.222.0.43 () {56 vars in 1120 bytes} [Wed Jul 20 19:11:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:34.197690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57108bc8-2696-4f1d-bc2a-9ea54639a8ba None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:34.291404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:34.307173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57108bc8-2696-4f1d-bc2a-9ea54639a8ba None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:34.357808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6d928-7991-47eb-9fa9-a066e3dfde18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:34.358736 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6d928-7991-47eb-9fa9-a066e3dfde18 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:34.359201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6d928-7991-47eb-9fa9-a066e3dfde18 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:34.359673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6d928-7991-47eb-9fa9-a066e3dfde18 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:34.360765 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6d928-7991-47eb-9fa9-a066e3dfde18 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:34.361280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6d928-7991-47eb-9fa9-a066e3dfde18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:34.362825 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6d928-7991-47eb-9fa9-a066e3dfde18 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:34.373030 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 466/928] 10.222.0.43 () {56 vars in 1115 bytes} [Wed Jul 20 19:11:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:34.378751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e99aa-add1-4c9c-be21-8e35c5676c1e None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:34.486866 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e99aa-add1-4c9c-be21-8e35c5676c1e None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:34.494639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:34.528681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5810e-3d99-4279-8714-df8b6726d1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:34.529489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5810e-3d99-4279-8714-df8b6726d1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:34.529769 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5810e-3d99-4279-8714-df8b6726d1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:34.530042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5810e-3d99-4279-8714-df8b6726d1e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:34.541078 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5810e-3d99-4279-8714-df8b6726d1e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:34.542509 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5810e-3d99-4279-8714-df8b6726d1e3 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:34.562123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-7fb5810e-3d99-4279-8714-df8b6726d1e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:34.562123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:34.562123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:34.562123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:34.562123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:34.562123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:34.562123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:34.562123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:34.562123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:34.562123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:34.562123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:34.562123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:34.562123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:34.562123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:34.562123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:34.563058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:34.563947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:34.563947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:34.563947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:34.563947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:34.563947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 463/929] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:34.570035 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6d928-7991-47eb-9fa9-a066e3dfde18 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:34.680870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6d928-7991-47eb-9fa9-a066e3dfde18 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:34.715867 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0077af-3189-437f-b70c-3e654e2d5b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:34.716556 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0077af-3189-437f-b70c-3e654e2d5b1c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:34.716833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0077af-3189-437f-b70c-3e654e2d5b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:34.717103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0077af-3189-437f-b70c-3e654e2d5b1c None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:34.717829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0077af-3189-437f-b70c-3e654e2d5b1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:34.718132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0077af-3189-437f-b70c-3e654e2d5b1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:34.719479 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0077af-3189-437f-b70c-3e654e2d5b1c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:34.732714 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 467/930] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:34.737946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5810e-3d99-4279-8714-df8b6726d1e3 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:34.844791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5810e-3d99-4279-8714-df8b6726d1e3 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:34.879732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13912887-28aa-4631-b89d-d1e49386ebc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:34.880575 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13912887-28aa-4631-b89d-d1e49386ebc8 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:34.880858 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13912887-28aa-4631-b89d-d1e49386ebc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:34.881131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13912887-28aa-4631-b89d-d1e49386ebc8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:34.888406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13912887-28aa-4631-b89d-d1e49386ebc8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:34.889906 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13912887-28aa-4631-b89d-d1e49386ebc8 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:34.905601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-13912887-28aa-4631-b89d-d1e49386ebc8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:11:34.905601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:34.905601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:34.905601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:34.905601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:34.905601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:34.905601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:34.905601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:34.905601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:34.905601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:34.905601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:34.905601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:34.905601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:34.905601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:34.905601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:34.905601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:34.906702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:34.907988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:34.907988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:34.907988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:34.907988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:34.907988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:34.907988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:34.907988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:34.907988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:34.907988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:11:34.907988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:34.907988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 464/931] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jul 20 19:11:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:34.913380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0077af-3189-437f-b70c-3e654e2d5b1c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:34.933175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:35.015039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0077af-3189-437f-b70c-3e654e2d5b1c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:35.050139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cf37e-5e86-4a16-8932-461bb45d72b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:35.050814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cf37e-5e86-4a16-8932-461bb45d72b5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:35.051098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cf37e-5e86-4a16-8932-461bb45d72b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:35.051422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cf37e-5e86-4a16-8932-461bb45d72b5 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:35.052213 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cf37e-5e86-4a16-8932-461bb45d72b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:35.052550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cf37e-5e86-4a16-8932-461bb45d72b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:35.054034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cf37e-5e86-4a16-8932-461bb45d72b5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:35.072370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 468/932] 10.222.0.43 () {56 vars in 1149 bytes} [Wed Jul 20 19:11:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:35.078022 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13912887-28aa-4631-b89d-d1e49386ebc8 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:35.099590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:35.140317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3b326-ec60-4d84-8117-e31b40c15345 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:35.140689 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3b326-ec60-4d84-8117-e31b40c15345 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:35.140886 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3b326-ec60-4d84-8117-e31b40c15345 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:35.142628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 469/933] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:35.148167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cf895-62c9-4ece-83a3-b0150dfe9100 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:35.148488 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cf895-62c9-4ece-83a3-b0150dfe9100 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:35.148765 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cf895-62c9-4ece-83a3-b0150dfe9100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:35.165912 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d53cf895-62c9-4ece-83a3-b0150dfe9100 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:35.167106 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d53cf895-62c9-4ece-83a3-b0150dfe9100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:35.187399 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13912887-28aa-4631-b89d-d1e49386ebc8 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:35.224925 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb71854-402a-4065-86f7-a910785a99d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:35.225726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb71854-402a-4065-86f7-a910785a99d8 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:35.226007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb71854-402a-4065-86f7-a910785a99d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:35.226291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb71854-402a-4065-86f7-a910785a99d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:35.236686 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:35.237159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb71854-402a-4065-86f7-a910785a99d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:35.238612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb71854-402a-4065-86f7-a910785a99d8 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:35.242826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 470/934] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:35.261794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-2fb71854-402a-4065-86f7-a910785a99d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:35.261794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:35.261794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:35.261794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:35.261794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:35.261794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:35.261794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:35.261794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:35.261794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:35.261794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:35.261794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:35.261794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:35.261794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:35.261794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:35.261794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:35.262839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:35.263718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:35.263718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:35.263718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:35.263718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:35.263718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 465/935] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:35.268903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cf895-62c9-4ece-83a3-b0150dfe9100 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:35.271302 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb71854-402a-4065-86f7-a910785a99d8 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:35.360159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:35.373746 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cf895-62c9-4ece-83a3-b0150dfe9100 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:35.405894 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb71854-402a-4065-86f7-a910785a99d8 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:35.418517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab564a-160e-4585-8bd7-99a3d66a980e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:35.419766 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab564a-160e-4585-8bd7-99a3d66a980e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:35.420594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab564a-160e-4585-8bd7-99a3d66a980e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:35.420720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab564a-160e-4585-8bd7-99a3d66a980e None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:35.421458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab564a-160e-4585-8bd7-99a3d66a980e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:35.421765 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab564a-160e-4585-8bd7-99a3d66a980e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:35.423130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab564a-160e-4585-8bd7-99a3d66a980e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:35.435724 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 471/936] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:35.441055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab564a-160e-4585-8bd7-99a3d66a980e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:35.448891 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739dc97-b6a9-4bb4-921a-fd8d72fd06c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:35.449894 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739dc97-b6a9-4bb4-921a-fd8d72fd06c8 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:35.450325 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739dc97-b6a9-4bb4-921a-fd8d72fd06c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:35.450746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739dc97-b6a9-4bb4-921a-fd8d72fd06c8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:35.451547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739dc97-b6a9-4bb4-921a-fd8d72fd06c8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:35.453668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739dc97-b6a9-4bb4-921a-fd8d72fd06c8 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-a739dc97-b6a9-4bb4-921a-fd8d72fd06c8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:35.470644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:35.471865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:35.471865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:35.471865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:35.471865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:35.471865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:35.471865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:35.471865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:35.471865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:35.471865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:35.471865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:35.471865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:35.471865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:35.471865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:35.471865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:35.471865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:35.471865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:35.473121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:35.473121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:35.473121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:35.473121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:35.473121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:35.473121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:35.473121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:35.473121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:35.473121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:35.473121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:35.473121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:35.473121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:35.473121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:35.473121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:35.476981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 466/937] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:35.486940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739dc97-b6a9-4bb4-921a-fd8d72fd06c8 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:35.516225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:35.572139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab564a-160e-4585-8bd7-99a3d66a980e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:35.609993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f571f-b068-40b4-b3ff-26efe532bb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:35.610807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f571f-b068-40b4-b3ff-26efe532bb69 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:35.611169 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f571f-b068-40b4-b3ff-26efe532bb69 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:35.611517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f571f-b068-40b4-b3ff-26efe532bb69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:35.616977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739dc97-b6a9-4bb4-921a-fd8d72fd06c8 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:35.620380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f571f-b068-40b4-b3ff-26efe532bb69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:35.621319 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f571f-b068-40b4-b3ff-26efe532bb69 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:35.637657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-860f571f-b068-40b4-b3ff-26efe532bb69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:35.637657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:35.637657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:35.637657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:35.637657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:35.637657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:35.637657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:35.637657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:35.637657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:35.637657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:35.637657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:35.637657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:35.637657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:35.637657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:35.637657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:35.638635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:35.639535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:35.639535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:35.639535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:35.639535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:35.639535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:35.639535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:35.639535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:35.639535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:35.643970 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 472/938] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:35.644942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f571f-b068-40b4-b3ff-26efe532bb69 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:35.660546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5a47a-fc7b-498e-aabb-81361380b1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:35.661789 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5a47a-fc7b-498e-aabb-81361380b1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:35.662726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5a47a-fc7b-498e-aabb-81361380b1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:35.663198 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5a47a-fc7b-498e-aabb-81361380b1e9 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:35.664391 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5a47a-fc7b-498e-aabb-81361380b1e9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:35.665145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5a47a-fc7b-498e-aabb-81361380b1e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:35.667057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5a47a-fc7b-498e-aabb-81361380b1e9 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:35.668422 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:35.679477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 467/939] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:35.680190 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5a47a-fc7b-498e-aabb-81361380b1e9 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:35.766336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f571f-b068-40b4-b3ff-26efe532bb69 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:35.785772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:35.801654 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5a47a-fc7b-498e-aabb-81361380b1e9 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:35.806382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d798d-ed59-4430-8545-7bc5d00d108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:35.807048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d798d-ed59-4430-8545-7bc5d00d108c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:35.807333 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d798d-ed59-4430-8545-7bc5d00d108c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:35.807865 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d798d-ed59-4430-8545-7bc5d00d108c None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:35.808697 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d798d-ed59-4430-8545-7bc5d00d108c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:35.809039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d798d-ed59-4430-8545-7bc5d00d108c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:35.810686 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d798d-ed59-4430-8545-7bc5d00d108c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:35.829550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 473/940] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:35.830494 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d798d-ed59-4430-8545-7bc5d00d108c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:35.839903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c627c-d9e4-4a37-9bbc-3ff75135a7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:35.840933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c627c-d9e4-4a37-9bbc-3ff75135a7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:35.841442 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c627c-d9e4-4a37-9bbc-3ff75135a7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:35.841932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c627c-d9e4-4a37-9bbc-3ff75135a7e9 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:35.842864 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c627c-d9e4-4a37-9bbc-3ff75135a7e9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:35.843372 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c627c-d9e4-4a37-9bbc-3ff75135a7e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:35.844991 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c627c-d9e4-4a37-9bbc-3ff75135a7e9 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:35.850488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 468/941] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:35 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:35.950942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d798d-ed59-4430-8545-7bc5d00d108c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:35.989909 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40107a57-d579-485b-9270-f689d482a8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:35.990821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40107a57-d579-485b-9270-f689d482a8e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:35.991104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40107a57-d579-485b-9270-f689d482a8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:35.991379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40107a57-d579-485b-9270-f689d482a8e8 None admin] PATH_INFO: `/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/2d514236db644010859fc18250f5894e/roles/35471af587764a54ba1c01a3ab111054` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:35.993804 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40107a57-d579-485b-9270-f689d482a8e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e2a50ebbb48417c911df499d98a0157, user_id=2d514236db644010859fc18250f5894e, role_id=35471af587764a54ba1c01a3ab111054)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:35.995317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40107a57-d579-485b-9270-f689d482a8e8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:36.011451 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 474/942] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:11:35 2022] PUT /identity/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/2d514236db644010859fc18250f5894e/roles/35471af587764a54ba1c01a3ab111054 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:11:36.102076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0f27a-51bc-40a6-a219-65b440149952 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:36.102649 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0f27a-51bc-40a6-a219-65b440149952 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:36.102649 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0f27a-51bc-40a6-a219-65b440149952 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:36.104232 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 469/943] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:36.111328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8b57f-a5a3-42a9-9dd1-019f6e9b3a8a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:36.111905 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8b57f-a5a3-42a9-9dd1-019f6e9b3a8a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:36.112431 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8b57f-a5a3-42a9-9dd1-019f6e9b3a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:36.135539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3fb8b57f-a5a3-42a9-9dd1-019f6e9b3a8a None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:36.136791 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3fb8b57f-a5a3-42a9-9dd1-019f6e9b3a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:36.165423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b06ae9-653c-46e6-b765-85195e9a8d6b None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:36.165738 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b06ae9-653c-46e6-b765-85195e9a8d6b None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:36.166025 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b06ae9-653c-46e6-b765-85195e9a8d6b None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:36.167402 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 470/944] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:36.172776 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd14ab-dbbc-4c0e-bbf9-169acef64efa None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:36.173061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd14ab-dbbc-4c0e-bbf9-169acef64efa None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:36.173342 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd14ab-dbbc-4c0e-bbf9-169acef64efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:36.178479 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:36.197143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4afd14ab-dbbc-4c0e-bbf9-169acef64efa None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:36.201050 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4afd14ab-dbbc-4c0e-bbf9-169acef64efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:36.234808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 475/945] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:36.258056 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8b57f-a5a3-42a9-9dd1-019f6e9b3a8a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:36.270058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:36.283008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 471/946] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:36.308458 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd14ab-dbbc-4c0e-bbf9-169acef64efa None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:36.393453 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8b57f-a5a3-42a9-9dd1-019f6e9b3a8a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:36.424645 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd14ab-dbbc-4c0e-bbf9-169acef64efa None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:36.442216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaed7b2-55f5-495b-a42a-ccacfad9e4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:36.443612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaed7b2-55f5-495b-a42a-ccacfad9e4e3 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:36.444138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaed7b2-55f5-495b-a42a-ccacfad9e4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:36.444647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaed7b2-55f5-495b-a42a-ccacfad9e4e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:36.445577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaed7b2-55f5-495b-a42a-ccacfad9e4e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:36.448023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaed7b2-55f5-495b-a42a-ccacfad9e4e3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:36.450595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: WARNING keystone.server.flask.common [None req-4eaed7b2-55f5-495b-a42a-ccacfad9e4e3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 19:11:36.465130 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e19e1-1a3e-4a49-8aa8-f224794e2784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:36.466264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e19e1-1a3e-4a49-8aa8-f224794e2784 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:36.466758 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e19e1-1a3e-4a49-8aa8-f224794e2784 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:36.467430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e19e1-1a3e-4a49-8aa8-f224794e2784 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:36.468872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e19e1-1a3e-4a49-8aa8-f224794e2784 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:36.470777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e19e1-1a3e-4a49-8aa8-f224794e2784 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:36.471749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.sql.core [None req-4eaed7b2-55f5-495b-a42a-ccacfad9e4e3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 19:11:36.471749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 19:11:36.471749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [parameters: {'id': '8003e30aa29949308c0598d0863e5f13', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 19:11:36.471749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80243) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-b19e19e1-1a3e-4a49-8aa8-f224794e2784 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:36.484561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:36.485667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:36.485667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:36.485667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:36.485667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:36.485667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:36.485667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:36.485667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:36.485667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:36.485667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:36.485667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:36.485667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:36.485667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:36.485667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:36.485667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:36.485667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:36.485667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:36.486507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:36.486507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:36.486507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:36.486507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:36.486507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:36.486507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:36.486507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:36.486507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:36.486507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:36.486507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:36.486507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:36.486507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:36.486507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:36.486507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:36.489493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 472/947] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:36.491916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-4eaed7b2-55f5-495b-a42a-ccacfad9e4e3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8003e30aa29949308c0598d0863e5f13.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8003e30aa29949308c0598d0863e5f13. Jul 20 19:11:36.491916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:36.491916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:11:36.491916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:11:36.491916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:11:36.491916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:11:36.491916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:11:36.491916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:11:36.491916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:11:36.491916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:11:36.491916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:11:36.491916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:11:36.491916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result.read() Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 20 19:11:36.492754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application yield resource Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise self.value Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application session.commit() Jul 20 19:11:36.493702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.session.flush() Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application compat.raise_( Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 19:11:36.494861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 19:11:36.495959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 19:11:36.495959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 19:11:36.495959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 19:11:36.495959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 19:11:36.495959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 19:11:36.495959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 19:11:36.495959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 19:11:36.495959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 19:11:36.495959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 19:11:36.495959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 19:11:36.495959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 19:11:36.495959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 19:11:36.495959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 19:11:36.495959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 19:11:36.495959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result.read() Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:11:36.496962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:11:36.498015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:11:36.498015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:11:36.498015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:11:36.498015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:11:36.498015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:11:36.498015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:11:36.498015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 19:11:36.498015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 19:11:36.498015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [parameters: {'id': '8003e30aa29949308c0598d0863e5f13', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 19:11:36.498015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 19:11:36.498015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:36.498015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:36.498015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:36.498015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:36.498015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:36.498971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:36.500103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:36.500103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 20 19:11:36.500103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 20 19:11:36.500103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:36.500103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:36.500103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 20 19:11:36.500103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 20 19:11:36.500103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8003e30aa29949308c0598d0863e5f13. Jul 20 19:11:36.500103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:36.502851 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e19e1-1a3e-4a49-8aa8-f224794e2784 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:36.507585 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 476/948] 10.222.0.43 () {60 vars in 1140 bytes} [Wed Jul 20 19:11:36 2022] POST /identity/v3/projects => generated 262 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:36.508813 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaed7b2-55f5-495b-a42a-ccacfad9e4e3 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:36.540840 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:36.569037 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:36.616591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e19e1-1a3e-4a49-8aa8-f224794e2784 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:36.623531 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaed7b2-55f5-495b-a42a-ccacfad9e4e3 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:36.653293 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b3ea8-04d1-4325-8def-886ebbb78956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:36.654120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b3ea8-04d1-4325-8def-886ebbb78956 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:36.654714 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b3ea8-04d1-4325-8def-886ebbb78956 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:36.654996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b3ea8-04d1-4325-8def-886ebbb78956 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:36.655959 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b3ea8-04d1-4325-8def-886ebbb78956 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:36.656372 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b3ea8-04d1-4325-8def-886ebbb78956 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:36.657651 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b3ea8-04d1-4325-8def-886ebbb78956 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:36.659547 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3896dd-86dd-46db-be4e-5e34ccc4fa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:36.660499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3896dd-86dd-46db-be4e-5e34ccc4fa45 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:36.660934 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3896dd-86dd-46db-be4e-5e34ccc4fa45 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:36.661311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3896dd-86dd-46db-be4e-5e34ccc4fa45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:36.662912 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 473/949] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:36.668342 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b3ea8-04d1-4325-8def-886ebbb78956 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:36.675483 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3896dd-86dd-46db-be4e-5e34ccc4fa45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:36.677474 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3896dd-86dd-46db-be4e-5e34ccc4fa45 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:36.702720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-3b3896dd-86dd-46db-be4e-5e34ccc4fa45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:36.702720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:36.702720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:36.702720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:36.702720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:36.702720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:36.702720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:36.702720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:36.702720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:36.702720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:36.702720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:36.702720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:36.702720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:36.702720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:36.702720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:36.703851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:36.704839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:36.704839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:36.704839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:36.704839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:36.704839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:36.704839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:36.704839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:36.704839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:36.706608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 477/950] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:36.712446 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3896dd-86dd-46db-be4e-5e34ccc4fa45 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:36.795600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b3ea8-04d1-4325-8def-886ebbb78956 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:36.817144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:36.819362 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3896dd-86dd-46db-be4e-5e34ccc4fa45 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:36.823380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:36.841989 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705a487-8a75-4e27-ba9d-150f4192d8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:36.842916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705a487-8a75-4e27-ba9d-150f4192d8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:36.843343 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705a487-8a75-4e27-ba9d-150f4192d8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:36.844146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705a487-8a75-4e27-ba9d-150f4192d8f1 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:36.845385 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705a487-8a75-4e27-ba9d-150f4192d8f1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:36.845854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705a487-8a75-4e27-ba9d-150f4192d8f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:36.848200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705a487-8a75-4e27-ba9d-150f4192d8f1 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:36.855074 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 474/951] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:36.860433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705a487-8a75-4e27-ba9d-150f4192d8f1 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:36.862370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9181b9-72a1-406c-9b27-7f5d5a97f8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:36.863032 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9181b9-72a1-406c-9b27-7f5d5a97f8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:36.863474 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9181b9-72a1-406c-9b27-7f5d5a97f8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:36.863724 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9181b9-72a1-406c-9b27-7f5d5a97f8e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:36.865301 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9181b9-72a1-406c-9b27-7f5d5a97f8e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:36.865301 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9181b9-72a1-406c-9b27-7f5d5a97f8e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:36.866366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9181b9-72a1-406c-9b27-7f5d5a97f8e0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:36.892712 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 478/952] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:36 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:36.975614 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705a487-8a75-4e27-ba9d-150f4192d8f1 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:37.015272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a52fe6-4abb-4362-8817-332d2af9cf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:37.016242 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a52fe6-4abb-4362-8817-332d2af9cf2b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:37.016701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a52fe6-4abb-4362-8817-332d2af9cf2b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:37.017134 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a52fe6-4abb-4362-8817-332d2af9cf2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:37.018100 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a52fe6-4abb-4362-8817-332d2af9cf2b None admin] RBAC: Adding query filter params (interface=public, service_id=dd365a18041f4f699133a0cf7450ee87, region_id=RegionOne) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:37.018567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a52fe6-4abb-4362-8817-332d2af9cf2b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:37.020911 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a52fe6-4abb-4362-8817-332d2af9cf2b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:37.027461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 475/953] 10.222.0.43 () {56 vars in 1251 bytes} [Wed Jul 20 19:11:36 2022] GET /identity/v3/endpoints?service_id=dd365a18041f4f699133a0cf7450ee87&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:37.033743 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9181b9-72a1-406c-9b27-7f5d5a97f8e0 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:37.142973 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9181b9-72a1-406c-9b27-7f5d5a97f8e0 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:37.180625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160fa94-0218-4a9f-8c24-0daae738765a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:37.181586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160fa94-0218-4a9f-8c24-0daae738765a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:37.182026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160fa94-0218-4a9f-8c24-0daae738765a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:37.182444 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160fa94-0218-4a9f-8c24-0daae738765a None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:37.183611 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160fa94-0218-4a9f-8c24-0daae738765a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:37.185191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160fa94-0218-4a9f-8c24-0daae738765a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:37.192799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 479/954] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jul 20 19:11:37 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:11:37.913167 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506e078-10c3-4bfe-b81e-b236a01b7ceb None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:37.913729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506e078-10c3-4bfe-b81e-b236a01b7ceb None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:37.913729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506e078-10c3-4bfe-b81e-b236a01b7ceb None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:37.915308 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 476/955] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:37.920751 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670acd6f-c194-4de6-bec5-dfe9d618af0f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:37.921153 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670acd6f-c194-4de6-bec5-dfe9d618af0f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:37.921338 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670acd6f-c194-4de6-bec5-dfe9d618af0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:37.938530 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-670acd6f-c194-4de6-bec5-dfe9d618af0f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:37.940064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-670acd6f-c194-4de6-bec5-dfe9d618af0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:37.942194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:38.009662 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 480/956] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:38.032450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506e078-10c3-4bfe-b81e-b236a01b7ceb None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:38.086672 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:38.122586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e08c1-16dc-4079-9f0e-aea025b8c344 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:38.122937 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e08c1-16dc-4079-9f0e-aea025b8c344 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:38.123157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e08c1-16dc-4079-9f0e-aea025b8c344 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:38.124808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 481/957] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:38.130395 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f2449-6040-4369-a8c1-a11c8b341bc0 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:38.130744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f2449-6040-4369-a8c1-a11c8b341bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:38.131026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f2449-6040-4369-a8c1-a11c8b341bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:38.145752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506e078-10c3-4bfe-b81e-b236a01b7ceb None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:38.148558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-675f2449-6040-4369-a8c1-a11c8b341bc0 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:38.149757 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-675f2449-6040-4369-a8c1-a11c8b341bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:38.188801 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12951074-54e6-4237-9a0c-d278734d8209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:38.189395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12951074-54e6-4237-9a0c-d278734d8209 None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:38.189681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12951074-54e6-4237-9a0c-d278734d8209 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:38.189957 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12951074-54e6-4237-9a0c-d278734d8209 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:38.190698 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12951074-54e6-4237-9a0c-d278734d8209 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:38.193036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12951074-54e6-4237-9a0c-d278734d8209 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:38.213648 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 477/958] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jul 20 19:11:38 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:38.242284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 482/959] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:38.268857 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12951074-54e6-4237-9a0c-d278734d8209 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:38.381192 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12951074-54e6-4237-9a0c-d278734d8209 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:38.391165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:38.417994 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcde54-742c-47e1-89e4-de0454db91f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:38.418808 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcde54-742c-47e1-89e4-de0454db91f0 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:38.419086 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcde54-742c-47e1-89e4-de0454db91f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:38.419353 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcde54-742c-47e1-89e4-de0454db91f0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:38.429789 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcde54-742c-47e1-89e4-de0454db91f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:38.438312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-1adcde54-742c-47e1-89e4-de0454db91f0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:38.438312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:38.438312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:38.438312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:38.438312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:38.438312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:38.438312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:38.438312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:38.438312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:38.438312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:38.438312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:38.438312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:38.438312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:38.438312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:38.438312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:38.438312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:38.439324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:38.439324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:38.439324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:38.439324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:38.439324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:38.439324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:38.439324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:38.439324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:38.439324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:38.439324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:38.439324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:38.439324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:38.439324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:38.439324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:38.439324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:38.439324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:38.440253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:38.440253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:38.440253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:38.440253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:38.440253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:38.440253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:38.440253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:38.440253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:38.440253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:38.440253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:38.440253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:38.440253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:38.440253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 478/960] 10.222.0.43 () {56 vars in 1120 bytes} [Wed Jul 20 19:11:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:38.446292 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f2449-6040-4369-a8c1-a11c8b341bc0 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:38.510212 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:38.550097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f2449-6040-4369-a8c1-a11c8b341bc0 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:38.585935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cd6d8-4d69-4604-8155-7b6d85f0ac8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:38.586601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cd6d8-4d69-4604-8155-7b6d85f0ac8e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:38.586936 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cd6d8-4d69-4604-8155-7b6d85f0ac8e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:38.587214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cd6d8-4d69-4604-8155-7b6d85f0ac8e None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:38.587962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81cd6d8-4d69-4604-8155-7b6d85f0ac8e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:38.588269 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81cd6d8-4d69-4604-8155-7b6d85f0ac8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:38.589655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81cd6d8-4d69-4604-8155-7b6d85f0ac8e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:38.598372 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 483/961] 10.222.0.43 () {56 vars in 1115 bytes} [Wed Jul 20 19:11:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:38.603873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcde54-742c-47e1-89e4-de0454db91f0 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:38.706918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcde54-742c-47e1-89e4-de0454db91f0 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:38.741737 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75c4e0-462b-46eb-92b2-895082a1ebeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:38.742564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75c4e0-462b-46eb-92b2-895082a1ebeb None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:38.742827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75c4e0-462b-46eb-92b2-895082a1ebeb None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:38.743115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75c4e0-462b-46eb-92b2-895082a1ebeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:38.754829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75c4e0-462b-46eb-92b2-895082a1ebeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:38.756273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75c4e0-462b-46eb-92b2-895082a1ebeb None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:38.775464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-5b75c4e0-462b-46eb-92b2-895082a1ebeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:38.775464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:38.775464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:38.775464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:38.775464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:38.775464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:38.775464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:38.775464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:38.775464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:38.775464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:38.775464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:38.775464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:38.775464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:38.775464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:38.775464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:38.776487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:38.777392 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:38.777392 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:38.777392 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:38.777392 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:38.777392 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 479/962] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:38.782586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cd6d8-4d69-4604-8155-7b6d85f0ac8e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:38.894047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cd6d8-4d69-4604-8155-7b6d85f0ac8e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:38.920899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:38.933116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae2398-666f-4547-9892-5c28fbf1f4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:38.934127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae2398-666f-4547-9892-5c28fbf1f4ba None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:38.934662 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae2398-666f-4547-9892-5c28fbf1f4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:38.935065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae2398-666f-4547-9892-5c28fbf1f4ba None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:38.936699 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae2398-666f-4547-9892-5c28fbf1f4ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:38.937226 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae2398-666f-4547-9892-5c28fbf1f4ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:38.939479 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae2398-666f-4547-9892-5c28fbf1f4ba None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:38.955596 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 484/963] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:38.961361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75c4e0-462b-46eb-92b2-895082a1ebeb None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:38.987266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:39.061468 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f3a08-301b-41bc-947f-d8bd2460fc70 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:39.061822 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f3a08-301b-41bc-947f-d8bd2460fc70 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:39.062022 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f3a08-301b-41bc-947f-d8bd2460fc70 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:39.063737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 485/964] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:39.069135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c83e2c-19d1-483b-bc78-d3044f7efc09 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:39.069441 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c83e2c-19d1-483b-bc78-d3044f7efc09 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:39.069722 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c83e2c-19d1-483b-bc78-d3044f7efc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:39.088386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75c4e0-462b-46eb-92b2-895082a1ebeb None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:39.088841 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b4c83e2c-19d1-483b-bc78-d3044f7efc09 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:39.090052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b4c83e2c-19d1-483b-bc78-d3044f7efc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:39.134065 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309398b4-cff4-43df-9119-f4cac6335807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:39.134896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309398b4-cff4-43df-9119-f4cac6335807 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:39.135176 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309398b4-cff4-43df-9119-f4cac6335807 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:39.135446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309398b4-cff4-43df-9119-f4cac6335807 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:39.143613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309398b4-cff4-43df-9119-f4cac6335807 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:39.145107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309398b4-cff4-43df-9119-f4cac6335807 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:39.163079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-309398b4-cff4-43df-9119-f4cac6335807 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:11:39.163079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:39.163079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:39.163079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:39.163079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:39.163079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:39.163079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:39.163079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:39.163079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:39.163079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:39.163079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:39.163079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:39.163079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:39.163079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:39.163079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:39.163079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:39.164323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:39.165633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:39.165633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:39.165633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:39.165633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:39.165633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:39.165633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:39.165633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:39.165633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:39.165633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:11:39.165633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:39.168436 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 480/965] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jul 20 19:11:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:39.170153 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309398b4-cff4-43df-9119-f4cac6335807 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:39.173352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 486/966] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:39.201627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c83e2c-19d1-483b-bc78-d3044f7efc09 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:39.267623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:39.289762 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309398b4-cff4-43df-9119-f4cac6335807 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:39.311781 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c83e2c-19d1-483b-bc78-d3044f7efc09 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:39.325948 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4977ca-dc6f-4a7a-bde9-ce8e42ce50f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:39.326809 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4977ca-dc6f-4a7a-bde9-ce8e42ce50f3 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:39.327253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4977ca-dc6f-4a7a-bde9-ce8e42ce50f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:39.327597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4977ca-dc6f-4a7a-bde9-ce8e42ce50f3 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:39.328563 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4977ca-dc6f-4a7a-bde9-ce8e42ce50f3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:39.328946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4977ca-dc6f-4a7a-bde9-ce8e42ce50f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:39.329809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:39.331565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4977ca-dc6f-4a7a-bde9-ce8e42ce50f3 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:39.350478 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac393782-6a6c-4fa4-a64e-2f310c0e55b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:39.351164 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac393782-6a6c-4fa4-a64e-2f310c0e55b8 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:39.351447 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac393782-6a6c-4fa4-a64e-2f310c0e55b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:39.351729 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac393782-6a6c-4fa4-a64e-2f310c0e55b8 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:39.352496 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac393782-6a6c-4fa4-a64e-2f310c0e55b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:39.353940 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac393782-6a6c-4fa4-a64e-2f310c0e55b8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:39.355040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 481/967] 10.222.0.43 () {56 vars in 1149 bytes} [Wed Jul 20 19:11:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:39.360483 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4977ca-dc6f-4a7a-bde9-ce8e42ce50f3 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:39.368809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.sql.core [None req-ac393782-6a6c-4fa4-a64e-2f310c0e55b8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 19:11:39.368809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 19:11:39.368809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [parameters: {'user_id': '88fbcbf517694cfabe888b4a7075a610', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 19:11:39.368809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80243) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 19:11:39.376628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-ac393782-6a6c-4fa4-a64e-2f310c0e55b8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 19:11:39.376628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:39.376628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:11:39.376628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:11:39.376628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:11:39.376628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:11:39.376628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:11:39.376628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:11:39.376628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:11:39.376628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:11:39.376628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:11:39.376628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:11:39.376628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:11:39.376628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result.read() Jul 20 19:11:39.376628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:11:39.377513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application yield resource Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise self.value Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application session.commit() Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 19:11:39.378499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.session.flush() Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application compat.raise_( Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 19:11:39.379416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 19:11:39.380317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 19:11:39.380317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 19:11:39.380317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 20 19:11:39.380317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 20 19:11:39.380317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 19:11:39.380317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 19:11:39.380317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 19:11:39.380317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 19:11:39.380317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 19:11:39.380317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 19:11:39.380317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 19:11:39.380317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 19:11:39.380317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 19:11:39.380317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 19:11:39.380317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:11:39.380317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception Jul 20 19:11:39.381155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:11:39.381155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:11:39.381155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:11:39.381155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:11:39.381155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:11:39.381155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:11:39.381155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:11:39.381155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:11:39.381155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:11:39.381155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:11:39.381155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:11:39.381155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result.read() Jul 20 19:11:39.381155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:11:39.381155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:11:39.381155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:11:39.381155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:11:39.382043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:11:39.382043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:11:39.382043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:11:39.382043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:11:39.382043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 19:11:39.382043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 19:11:39.382043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [parameters: {'user_id': '88fbcbf517694cfabe888b4a7075a610', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 19:11:39.382043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 19:11:39.382043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:39.382043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:39.382043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:39.382043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:39.382043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:39.382043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:39.382043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 20 19:11:39.382845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 20 19:11:39.383737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 20 19:11:39.383737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 20 19:11:39.383737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 19:11:39.383737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 19:11:39.383737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 19:11:39.383737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:39.383737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 487/968] 10.222.0.43 () {60 vars in 1131 bytes} [Wed Jul 20 19:11:39 2022] POST /identity/v3/users => generated 160 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:39.383737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a465468-c930-4b9a-9a22-5547ca7af0c5 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:39.383737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a465468-c930-4b9a-9a22-5547ca7af0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:39.383737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a465468-c930-4b9a-9a22-5547ca7af0c5 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:39.385952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 488/969] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:39.385952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a465468-c930-4b9a-9a22-5547ca7af0c5 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:39.489716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4977ca-dc6f-4a7a-bde9-ce8e42ce50f3 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:39.502781 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a465468-c930-4b9a-9a22-5547ca7af0c5 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:39.528653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa0fbf-d43a-40b7-9f1a-4ece566edc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:39.529576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa0fbf-d43a-40b7-9f1a-4ece566edc8c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:39.530053 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa0fbf-d43a-40b7-9f1a-4ece566edc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:39.530502 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa0fbf-d43a-40b7-9f1a-4ece566edc8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:39.539068 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848151b-f6b5-46cc-9a88-5695d30a43ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:39.539785 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:39.540264 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848151b-f6b5-46cc-9a88-5695d30a43ed None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:39.540574 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848151b-f6b5-46cc-9a88-5695d30a43ed None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:39.540859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848151b-f6b5-46cc-9a88-5695d30a43ed None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:39.549308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848151b-f6b5-46cc-9a88-5695d30a43ed None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:39.550875 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848151b-f6b5-46cc-9a88-5695d30a43ed None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:39.565717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-8848151b-f6b5-46cc-9a88-5695d30a43ed None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:11:39.565717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:39.565717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:39.565717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:39.565717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:39.565717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:39.565717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:39.565717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:39.565717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:39.565717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:39.565717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:39.565717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:39.565717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:39.565717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:39.565717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:39.565717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:39.566727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:39.567673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:39.567673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:39.567673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:39.567673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:39.567673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:39.567673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:39.567673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:39.567673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:39.567673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:11:39.567673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:39.567673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 489/970] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:39.568268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f904b9-35d4-4a55-be91-f6d89d708883 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:39.568706 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f904b9-35d4-4a55-be91-f6d89d708883 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:39.568706 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f904b9-35d4-4a55-be91-f6d89d708883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:39.585932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a7f904b9-35d4-4a55-be91-f6d89d708883 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:39.587449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a7f904b9-35d4-4a55-be91-f6d89d708883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:39.618184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa0fbf-d43a-40b7-9f1a-4ece566edc8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:39.618873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:39.620440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa0fbf-d43a-40b7-9f1a-4ece566edc8c None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:39.664406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 490/971] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:39.664406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f904b9-35d4-4a55-be91-f6d89d708883 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:39.733197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 482/972] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:39 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:11:39.734267 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa0fbf-d43a-40b7-9f1a-4ece566edc8c None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:39.777042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:39.779526 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f904b9-35d4-4a55-be91-f6d89d708883 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:39.815212 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc6ac4-2bf6-4ee3-812d-4f763d38e0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:39.815875 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc6ac4-2bf6-4ee3-812d-4f763d38e0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:39.816156 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc6ac4-2bf6-4ee3-812d-4f763d38e0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:39.816968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc6ac4-2bf6-4ee3-812d-4f763d38e0d1 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:39.817913 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc6ac4-2bf6-4ee3-812d-4f763d38e0d1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:39.818084 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc6ac4-2bf6-4ee3-812d-4f763d38e0d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:39.819508 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc6ac4-2bf6-4ee3-812d-4f763d38e0d1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:39.845268 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa0fbf-d43a-40b7-9f1a-4ece566edc8c None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:39.864323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 491/973] 10.222.0.43 () {56 vars in 1101 bytes} [Wed Jul 20 19:11:39 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:39.864323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc6ac4-2bf6-4ee3-812d-4f763d38e0d1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:39.897170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2362df-b7a9-4e0a-a278-d0e89b9a7a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:39.898094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2362df-b7a9-4e0a-a278-d0e89b9a7a51 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:39.898444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2362df-b7a9-4e0a-a278-d0e89b9a7a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:39.898799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2362df-b7a9-4e0a-a278-d0e89b9a7a51 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:39.899532 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2362df-b7a9-4e0a-a278-d0e89b9a7a51 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:39.901080 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2362df-b7a9-4e0a-a278-d0e89b9a7a51 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:39.902125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-8b2362df-b7a9-4e0a-a278-d0e89b9a7a51 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:39.916170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:39.917357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:39.917357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:39.917357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:39.917357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:39.917357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:39.917357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:39.917357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:39.917357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:39.917357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:39.917357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:39.917357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:39.917357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:39.917357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:39.917357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:39.917357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:39.917357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:39.918246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:39.918246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:39.918246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:39.918246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:39.918246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:39.918246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:39.918246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:39.918246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:39.918246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:39.918246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:39.918246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:39.918246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:39.918246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:39.918246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:39.923022 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 483/974] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:39.924109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2362df-b7a9-4e0a-a278-d0e89b9a7a51 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:40.004319 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc6ac4-2bf6-4ee3-812d-4f763d38e0d1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:40.047869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2362df-b7a9-4e0a-a278-d0e89b9a7a51 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:40.048388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e993e-ea04-4353-9984-61cdb955e2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:40.048787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e993e-ea04-4353-9984-61cdb955e2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:40.049074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e993e-ea04-4353-9984-61cdb955e2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:40.049371 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e993e-ea04-4353-9984-61cdb955e2c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:40.049854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:40.066964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e993e-ea04-4353-9984-61cdb955e2c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:40.068536 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e993e-ea04-4353-9984-61cdb955e2c1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:40.088844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-5a5e993e-ea04-4353-9984-61cdb955e2c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:40.088844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:40.088844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:40.088844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:40.088844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:40.088844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:40.088844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:40.088844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:40.088844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:40.088844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:40.088844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:40.088844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:40.088844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:40.088844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:40.088844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:40.089787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:40.091906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:40.091906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:40.091906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:40.091906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:40.091906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 492/975] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:40.095869 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e993e-ea04-4353-9984-61cdb955e2c1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:40.096007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa321db-c362-4f2a-af77-1c02eb42a164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:40.096671 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa321db-c362-4f2a-af77-1c02eb42a164 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:40.096960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa321db-c362-4f2a-af77-1c02eb42a164 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:40.097240 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa321db-c362-4f2a-af77-1c02eb42a164 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:40.098004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa321db-c362-4f2a-af77-1c02eb42a164 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:40.098318 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa321db-c362-4f2a-af77-1c02eb42a164 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:40.099689 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa321db-c362-4f2a-af77-1c02eb42a164 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:40.106157 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 484/976] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:40.113966 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa321db-c362-4f2a-af77-1c02eb42a164 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:40.190508 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:40.213830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e993e-ea04-4353-9984-61cdb955e2c1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:40.241497 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa321db-c362-4f2a-af77-1c02eb42a164 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:40.252259 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b349a8e-cb31-4588-9a5d-a48c5c46916c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:40.252960 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b349a8e-cb31-4588-9a5d-a48c5c46916c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:40.253608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b349a8e-cb31-4588-9a5d-a48c5c46916c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:40.253932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b349a8e-cb31-4588-9a5d-a48c5c46916c None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:40.254983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b349a8e-cb31-4588-9a5d-a48c5c46916c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:40.255306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b349a8e-cb31-4588-9a5d-a48c5c46916c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:40.256881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b349a8e-cb31-4588-9a5d-a48c5c46916c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:40.268894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 493/977] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:40.274904 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b349a8e-cb31-4588-9a5d-a48c5c46916c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:40.287257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bef47-ee20-4790-b888-18b544841fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:40.288251 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bef47-ee20-4790-b888-18b544841fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:40.288579 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bef47-ee20-4790-b888-18b544841fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:40.288860 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bef47-ee20-4790-b888-18b544841fd4 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:40.289584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6bef47-ee20-4790-b888-18b544841fd4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:40.289895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6bef47-ee20-4790-b888-18b544841fd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:40.291254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6bef47-ee20-4790-b888-18b544841fd4 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:40.296313 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 485/978] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:40.303094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bef47-ee20-4790-b888-18b544841fd4 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:40.333642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:40.390888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b349a8e-cb31-4588-9a5d-a48c5c46916c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:40.415113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bef47-ee20-4790-b888-18b544841fd4 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:40.431304 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71685e66-7e24-4790-9375-f49a20748738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:40.432103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71685e66-7e24-4790-9375-f49a20748738 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:40.432421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71685e66-7e24-4790-9375-f49a20748738 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:40.432952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71685e66-7e24-4790-9375-f49a20748738 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:40.439553 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71685e66-7e24-4790-9375-f49a20748738 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:40.441024 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71685e66-7e24-4790-9375-f49a20748738 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:40.442733 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:40.450257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510aed4b-afb3-4456-a3f4-e9aa07cb096d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:40.450929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aed4b-afb3-4456-a3f4-e9aa07cb096d None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:40.451210 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aed4b-afb3-4456-a3f4-e9aa07cb096d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:40.451485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aed4b-afb3-4456-a3f4-e9aa07cb096d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:40.452227 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510aed4b-afb3-4456-a3f4-e9aa07cb096d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:40.453603 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510aed4b-afb3-4456-a3f4-e9aa07cb096d None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:40.455928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-71685e66-7e24-4790-9375-f49a20748738 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:40.455928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:40.455928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:40.455928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:40.455928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:40.455928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:40.455928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:40.455928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:40.455928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:40.455928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:40.455928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:40.455928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:40.455928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:40.455928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:40.455928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:40.456958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:40.457830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:40.457830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:40.457830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:40.457830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:40.457830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:40.457830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:40.457830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:40.457830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:40.457830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 494/979] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:40.458723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08769d99-6713-4e94-8a32-a4013d1d996d None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:40.459012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08769d99-6713-4e94-8a32-a4013d1d996d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:40.459292 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08769d99-6713-4e94-8a32-a4013d1d996d None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:40.460586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 495/980] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:40.462736 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08769d99-6713-4e94-8a32-a4013d1d996d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:40.477050 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 486/981] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jul 20 19:11:40 2022] POST /identity/v3/endpoints => generated 349 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:11:40.478167 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79855d74-f728-4e51-b1c5-60c8ade3ada1 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:40.478455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79855d74-f728-4e51-b1c5-60c8ade3ada1 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:40.478729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79855d74-f728-4e51-b1c5-60c8ade3ada1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:40.496731 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-79855d74-f728-4e51-b1c5-60c8ade3ada1 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:40.498297 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-79855d74-f728-4e51-b1c5-60c8ade3ada1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:40.584147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 487/982] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:40.588923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08769d99-6713-4e94-8a32-a4013d1d996d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:40.607664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79855d74-f728-4e51-b1c5-60c8ade3ada1 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:40.623138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:40.630943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbff15-6cc1-4441-bf1e-f80012c521e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:40.632063 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbff15-6cc1-4441-bf1e-f80012c521e6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:40.632463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbff15-6cc1-4441-bf1e-f80012c521e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:40.632746 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbff15-6cc1-4441-bf1e-f80012c521e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:40.634913 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dbff15-6cc1-4441-bf1e-f80012c521e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:40.635184 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dbff15-6cc1-4441-bf1e-f80012c521e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:40.636810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dbff15-6cc1-4441-bf1e-f80012c521e6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:40.649741 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 496/983] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:40.655252 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbff15-6cc1-4441-bf1e-f80012c521e6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:40.715899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:40.723179 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79855d74-f728-4e51-b1c5-60c8ade3ada1 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:40.759868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780525e-c0b7-4cfa-8ef8-c9b893c726dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:40.760691 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780525e-c0b7-4cfa-8ef8-c9b893c726dd None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:40.760977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780525e-c0b7-4cfa-8ef8-c9b893c726dd None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:40.761253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780525e-c0b7-4cfa-8ef8-c9b893c726dd None admin] PATH_INFO: `/v3/services/compute` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:40.761918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780525e-c0b7-4cfa-8ef8-c9b893c726dd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:40.763341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780525e-c0b7-4cfa-8ef8-c9b893c726dd None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:40.763619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbff15-6cc1-4441-bf1e-f80012c521e6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-e780525e-c0b7-4cfa-8ef8-c9b893c726dd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:40.776077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:40.777091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:40.777091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:40.777091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:40.777091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:40.777091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:40.777091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:40.777091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:40.777091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:40.777091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:40.777091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:40.777091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:40.777091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:40.777091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:40.777091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:40.777091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:40.777091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:40.778663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:40.778663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:40.778663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:40.778663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:40.778663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:40.778663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:40.778663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:40.778663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:40.778663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:40.778663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:40.778663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:40.778663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:40.778663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:40.778663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:40.778663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 488/984] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:40.783078 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780525e-c0b7-4cfa-8ef8-c9b893c726dd None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:40.804038 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ca0b7-a4f9-43c3-9a0b-acb6dfbfa55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:40.804732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ca0b7-a4f9-43c3-9a0b-acb6dfbfa55e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:40.805021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ca0b7-a4f9-43c3-9a0b-acb6dfbfa55e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:40.805306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ca0b7-a4f9-43c3-9a0b-acb6dfbfa55e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:40.806154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ca0b7-a4f9-43c3-9a0b-acb6dfbfa55e None admin] RBAC: Adding query filter params (role.id=35471af587764a54ba1c01a3ab111054, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=2d514236db644010859fc18250f5894e) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:40.806472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ca0b7-a4f9-43c3-9a0b-acb6dfbfa55e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:40.808182 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ca0b7-a4f9-43c3-9a0b-acb6dfbfa55e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:40.814681 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 497/985] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:40 2022] GET /identity/v3/role_assignments?user.id=2d514236db644010859fc18250f5894e&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=35471af587764a54ba1c01a3ab111054 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:40.878042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:40.905367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780525e-c0b7-4cfa-8ef8-c9b893c726dd None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:40.943049 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555aa773-7438-4951-85b1-62fee6fb6110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:40.943737 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555aa773-7438-4951-85b1-62fee6fb6110 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:40.944020 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555aa773-7438-4951-85b1-62fee6fb6110 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:40.944322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555aa773-7438-4951-85b1-62fee6fb6110 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:40.945069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555aa773-7438-4951-85b1-62fee6fb6110 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:40.945377 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555aa773-7438-4951-85b1-62fee6fb6110 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:40.946765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555aa773-7438-4951-85b1-62fee6fb6110 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:40.954101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 489/986] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:40.959534 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ca0b7-a4f9-43c3-9a0b-acb6dfbfa55e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:41.071831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ca0b7-a4f9-43c3-9a0b-acb6dfbfa55e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:41.095215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:41.109154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a4abc-b87d-4e1a-b479-60954b219e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:41.109831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a4abc-b87d-4e1a-b479-60954b219e77 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:41.110116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a4abc-b87d-4e1a-b479-60954b219e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:41.110396 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a4abc-b87d-4e1a-b479-60954b219e77 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:41.111115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a4abc-b87d-4e1a-b479-60954b219e77 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:41.111426 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a4abc-b87d-4e1a-b479-60954b219e77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:41.112863 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a4abc-b87d-4e1a-b479-60954b219e77 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:41.118670 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 498/987] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:41.126164 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555aa773-7438-4951-85b1-62fee6fb6110 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:41.242041 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555aa773-7438-4951-85b1-62fee6fb6110 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:41.278944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd358c-94f4-416b-9dc1-0a0ad4f60dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:41.279618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd358c-94f4-416b-9dc1-0a0ad4f60dce None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:41.279898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd358c-94f4-416b-9dc1-0a0ad4f60dce None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:41.280174 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd358c-94f4-416b-9dc1-0a0ad4f60dce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:41.281004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd358c-94f4-416b-9dc1-0a0ad4f60dce None admin] RBAC: Adding query filter params (interface=public, service_id=ce2f5aa321b5423d890c311375867d17, region_id=RegionOne) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:41.281322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd358c-94f4-416b-9dc1-0a0ad4f60dce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:41.282681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd358c-94f4-416b-9dc1-0a0ad4f60dce None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:41.293583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 490/988] 10.222.0.43 () {56 vars in 1251 bytes} [Wed Jul 20 19:11:41 2022] GET /identity/v3/endpoints?service_id=ce2f5aa321b5423d890c311375867d17&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:41.294901 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a4abc-b87d-4e1a-b479-60954b219e77 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:41.411325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a4abc-b87d-4e1a-b479-60954b219e77 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:41.445671 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd185e-c8e0-419a-a93c-59ab2566f837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:41.446333 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd185e-c8e0-419a-a93c-59ab2566f837 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:41.446615 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd185e-c8e0-419a-a93c-59ab2566f837 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:41.446891 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd185e-c8e0-419a-a93c-59ab2566f837 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:41.447576 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd185e-c8e0-419a-a93c-59ab2566f837 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:41.448969 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd185e-c8e0-419a-a93c-59ab2566f837 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:41.460316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 499/989] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jul 20 19:11:41 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:11:42.541117 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9225f3-445d-4c5b-98dd-db130ae5c87a None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:42.541671 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9225f3-445d-4c5b-98dd-db130ae5c87a None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:42.541671 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9225f3-445d-4c5b-98dd-db130ae5c87a None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:42.543180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 491/990] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:42.550296 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6754b-4ec0-4cac-9957-9e70bd31a78b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:42.551236 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6754b-4ec0-4cac-9957-9e70bd31a78b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:42.551444 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6754b-4ec0-4cac-9957-9e70bd31a78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:42.569678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-9de6754b-4ec0-4cac-9957-9e70bd31a78b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:42.570951 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9de6754b-4ec0-4cac-9957-9e70bd31a78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:42.620969 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:42.642961 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 500/991] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:42.666777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9225f3-445d-4c5b-98dd-db130ae5c87a None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:42.670643 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:42.771296 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9225f3-445d-4c5b-98dd-db130ae5c87a None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:42.805489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b95bb-8b27-4fc9-a3b4-2f69d0046cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:42.806296 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b95bb-8b27-4fc9-a3b4-2f69d0046cce None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:42.806580 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b95bb-8b27-4fc9-a3b4-2f69d0046cce None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:42.806856 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b95bb-8b27-4fc9-a3b4-2f69d0046cce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:42.807772 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b95bb-8b27-4fc9-a3b4-2f69d0046cce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:42.809604 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b95bb-8b27-4fc9-a3b4-2f69d0046cce None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-b02b95bb-8b27-4fc9-a3b4-2f69d0046cce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:42.821262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:42.822219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:42.822219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:42.822219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:42.822219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:42.822219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:42.822219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:42.822219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:42.822219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:42.822219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:42.822219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:42.822219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:42.822219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:42.822219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:42.822219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:42.822219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:42.822219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:42.823055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:42.823055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:42.823055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:42.823055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:42.823055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:42.823055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:42.823055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:42.823055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:42.823055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:42.823055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:42.823055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:42.823055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:42.823055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:42.823055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:42.827524 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 492/992] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:42.828377 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6754b-4ec0-4cac-9957-9e70bd31a78b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:42.927539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6754b-4ec0-4cac-9957-9e70bd31a78b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:42.930914 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6910fd-24e7-4d3b-a6c9-c5bb8d9f6fb3 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:42.931206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6910fd-24e7-4d3b-a6c9-c5bb8d9f6fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:42.931486 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6910fd-24e7-4d3b-a6c9-c5bb8d9f6fb3 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:42.933279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 493/993] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:42.938052 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d65751-ab33-49e9-b8fd-b4a8b2c29fdd None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:42.938338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d65751-ab33-49e9-b8fd-b4a8b2c29fdd None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:42.938618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d65751-ab33-49e9-b8fd-b4a8b2c29fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:42.955780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-20d65751-ab33-49e9-b8fd-b4a8b2c29fdd None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:42.957073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-20d65751-ab33-49e9-b8fd-b4a8b2c29fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:42.967116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935fad9-64da-4290-a52c-1532d83926bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:42.967792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935fad9-64da-4290-a52c-1532d83926bf None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:42.968081 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935fad9-64da-4290-a52c-1532d83926bf None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:42.968389 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935fad9-64da-4290-a52c-1532d83926bf None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:42.969114 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935fad9-64da-4290-a52c-1532d83926bf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:42.969423 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935fad9-64da-4290-a52c-1532d83926bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:42.970796 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935fad9-64da-4290-a52c-1532d83926bf None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:42.980869 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 501/994] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:42.981886 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935fad9-64da-4290-a52c-1532d83926bf None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:43.023713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:43.029068 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:43.030436 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 494/995] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:43.053652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d65751-ab33-49e9-b8fd-b4a8b2c29fdd None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:43.086725 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935fad9-64da-4290-a52c-1532d83926bf None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:43.124694 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e4354-2739-4015-8aa1-9e2120cbd825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:43.125365 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e4354-2739-4015-8aa1-9e2120cbd825 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:43.125647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e4354-2739-4015-8aa1-9e2120cbd825 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:43.125920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e4354-2739-4015-8aa1-9e2120cbd825 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:43.126636 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e4354-2739-4015-8aa1-9e2120cbd825 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:43.126943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e4354-2739-4015-8aa1-9e2120cbd825 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:43.128791 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e4354-2739-4015-8aa1-9e2120cbd825 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:43.134608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 502/996] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:43.140404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e4354-2739-4015-8aa1-9e2120cbd825 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:43.160977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d65751-ab33-49e9-b8fd-b4a8b2c29fdd None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:43.198854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9193a7-b5e7-4967-aaf2-2effbbb685ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:43.199577 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9193a7-b5e7-4967-aaf2-2effbbb685ac None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:43.199863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9193a7-b5e7-4967-aaf2-2effbbb685ac None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:43.200142 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9193a7-b5e7-4967-aaf2-2effbbb685ac None admin] PATH_INFO: `/v3/services/image` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:43.200832 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9193a7-b5e7-4967-aaf2-2effbbb685ac None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:43.202202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9193a7-b5e7-4967-aaf2-2effbbb685ac None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-6c9193a7-b5e7-4967-aaf2-2effbbb685ac None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:43.217255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:43.218549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:43.218549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:43.218549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:43.218549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:43.218549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:43.218549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:43.218549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:43.218549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:43.218549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:43.218549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:43.218549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:43.218549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:43.218549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:43.218549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:43.218549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:43.218549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:43.219380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:43.219380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:43.219380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:43.219380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:43.219380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:43.219380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:43.219380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:43.219380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:43.219380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:43.219380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:43.219380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:43.219380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:43.219380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:43.219380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:43.219380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 495/997] 10.222.0.43 () {56 vars in 1105 bytes} [Wed Jul 20 19:11:43 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:43.224898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9193a7-b5e7-4967-aaf2-2effbbb685ac None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:43.260973 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e4354-2739-4015-8aa1-9e2120cbd825 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:43.273777 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:43.299278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:43.303836 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ed0e3-77a6-4712-a7af-f73440e00a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:43.304755 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ed0e3-77a6-4712-a7af-f73440e00a36 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:43.305043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ed0e3-77a6-4712-a7af-f73440e00a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:43.305329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ed0e3-77a6-4712-a7af-f73440e00a36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:43.306141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ed0e3-77a6-4712-a7af-f73440e00a36 None admin] RBAC: Adding query filter params (interface=admin, service_id=dd365a18041f4f699133a0cf7450ee87, region_id=RegionOne) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:43.306460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ed0e3-77a6-4712-a7af-f73440e00a36 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:43.308097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ed0e3-77a6-4712-a7af-f73440e00a36 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:43.318015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 503/998] 10.222.0.43 () {56 vars in 1249 bytes} [Wed Jul 20 19:11:43 2022] GET /identity/v3/endpoints?service_id=dd365a18041f4f699133a0cf7450ee87&interface=admin®ion_id=RegionOne => generated 193 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:43.319322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ed0e3-77a6-4712-a7af-f73440e00a36 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:43.344897 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9193a7-b5e7-4967-aaf2-2effbbb685ac None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:43.382946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2b7d8-0d74-4b82-ab37-4b6a4a94574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:43.383985 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2b7d8-0d74-4b82-ab37-4b6a4a94574b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:43.384269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2b7d8-0d74-4b82-ab37-4b6a4a94574b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:43.384573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2b7d8-0d74-4b82-ab37-4b6a4a94574b None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:43.385288 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2b7d8-0d74-4b82-ab37-4b6a4a94574b None admin] RBAC: Adding query filter params (name=image) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:43.385597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2b7d8-0d74-4b82-ab37-4b6a4a94574b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:43.387610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2b7d8-0d74-4b82-ab37-4b6a4a94574b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:43.397701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 496/999] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:43.398948 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2b7d8-0d74-4b82-ab37-4b6a4a94574b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:43.436757 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ed0e3-77a6-4712-a7af-f73440e00a36 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:43.472702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf7f36-09be-4756-9104-8df6365a3be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:43.473284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7f36-09be-4756-9104-8df6365a3be2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:43.473567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7f36-09be-4756-9104-8df6365a3be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:43.473849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7f36-09be-4756-9104-8df6365a3be2 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:43.474517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf7f36-09be-4756-9104-8df6365a3be2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:43.476328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf7f36-09be-4756-9104-8df6365a3be2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:43.482501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 504/1000] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jul 20 19:11:43 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:11:43.507321 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a211a-0ad2-4519-bf2e-0630c447047e None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:43.507645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a211a-0ad2-4519-bf2e-0630c447047e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:43.507866 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a211a-0ad2-4519-bf2e-0630c447047e None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:43.509651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 505/1001] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:43.515375 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf194dd4-9841-4e39-897c-1b89aaac0ba9 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:43.515772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf194dd4-9841-4e39-897c-1b89aaac0ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:43.515946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf194dd4-9841-4e39-897c-1b89aaac0ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:43.520367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2b7d8-0d74-4b82-ab37-4b6a4a94574b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:43.533720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-cf194dd4-9841-4e39-897c-1b89aaac0ba9 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:43.534982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cf194dd4-9841-4e39-897c-1b89aaac0ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:43.553081 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:43.564180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2f098-59be-4ff3-9ad5-e07eac625d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:43.564919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2f098-59be-4ff3-9ad5-e07eac625d24 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:43.565222 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2f098-59be-4ff3-9ad5-e07eac625d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:43.565478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2f098-59be-4ff3-9ad5-e07eac625d24 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:43.566765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da2f098-59be-4ff3-9ad5-e07eac625d24 None admin] RBAC: Adding query filter params (type=image) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:43.567069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da2f098-59be-4ff3-9ad5-e07eac625d24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:43.568506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da2f098-59be-4ff3-9ad5-e07eac625d24 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:43.569655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:43.574581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 497/1002] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:43 2022] GET /identity/v3/services?type=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:43.621708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 506/1003] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:43 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:43.645813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2f098-59be-4ff3-9ad5-e07eac625d24 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:43.759398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2f098-59be-4ff3-9ad5-e07eac625d24 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:43.796918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8573b44-a0ad-4e8a-9271-6744f88aaf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:43.797725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8573b44-a0ad-4e8a-9271-6744f88aaf85 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:43.798008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8573b44-a0ad-4e8a-9271-6744f88aaf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:43.798286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8573b44-a0ad-4e8a-9271-6744f88aaf85 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:43.798943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8573b44-a0ad-4e8a-9271-6744f88aaf85 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:43.800326 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8573b44-a0ad-4e8a-9271-6744f88aaf85 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-b8573b44-a0ad-4e8a-9271-6744f88aaf85 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:43.813886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:43.814904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:43.814904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:43.814904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:43.814904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:43.814904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:43.814904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:43.814904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:43.814904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:43.814904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:43.814904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:43.814904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:43.814904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:43.814904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:43.814904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:43.814904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:43.814904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:43.815730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:43.815730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:43.815730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:43.815730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:43.815730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:43.815730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:43.815730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:43.815730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:43.815730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:43.815730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:43.815730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:43.815730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:43.815730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:43.815730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:43.820355 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 498/1004] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:43.821889 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf194dd4-9841-4e39-897c-1b89aaac0ba9 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:43.921832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf194dd4-9841-4e39-897c-1b89aaac0ba9 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:43.957413 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a631355-0065-4b33-ae3b-7f720c52ad70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:43.958088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a631355-0065-4b33-ae3b-7f720c52ad70 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:43.958370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a631355-0065-4b33-ae3b-7f720c52ad70 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:43.958646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a631355-0065-4b33-ae3b-7f720c52ad70 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:43.959354 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a631355-0065-4b33-ae3b-7f720c52ad70 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:43.959663 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a631355-0065-4b33-ae3b-7f720c52ad70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:43.961047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a631355-0065-4b33-ae3b-7f720c52ad70 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:43.966254 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 507/1005] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:43.971910 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8573b44-a0ad-4e8a-9271-6744f88aaf85 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:44.030076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:44.078171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8573b44-a0ad-4e8a-9271-6744f88aaf85 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:44.111586 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff3076-af08-46f4-a18e-3a7e6c544caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:44.112259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff3076-af08-46f4-a18e-3a7e6c544caf None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:44.112571 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff3076-af08-46f4-a18e-3a7e6c544caf None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:44.112849 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff3076-af08-46f4-a18e-3a7e6c544caf None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:44.114103 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff3076-af08-46f4-a18e-3a7e6c544caf None admin] RBAC: Adding query filter params (type=compute) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:44.114413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff3076-af08-46f4-a18e-3a7e6c544caf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:44.115771 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff3076-af08-46f4-a18e-3a7e6c544caf None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:44.125484 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 499/1006] 10.222.0.43 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:43 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:44.127058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a631355-0065-4b33-ae3b-7f720c52ad70 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:44.159030 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:44.233505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a631355-0065-4b33-ae3b-7f720c52ad70 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:44.267926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f4ec6-759f-4324-8393-04d7dd6e2b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:44.268617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f4ec6-759f-4324-8393-04d7dd6e2b8f None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:44.268900 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f4ec6-759f-4324-8393-04d7dd6e2b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:44.269173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f4ec6-759f-4324-8393-04d7dd6e2b8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:44.270073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f4ec6-759f-4324-8393-04d7dd6e2b8f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:44.271433 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f4ec6-759f-4324-8393-04d7dd6e2b8f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:44.291998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 508/1007] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jul 20 19:11:44 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:45.536248 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f16c7-af34-4b13-868a-14cab9d8ab35 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:45.536781 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f16c7-af34-4b13-868a-14cab9d8ab35 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:45.536893 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f16c7-af34-4b13-868a-14cab9d8ab35 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:45.538247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 500/1008] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:45.543333 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f9f69-b833-466e-bdc8-6d4480584c09 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:45.543754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f9f69-b833-466e-bdc8-6d4480584c09 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:45.543928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f9f69-b833-466e-bdc8-6d4480584c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:45.561187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2d8f9f69-b833-466e-bdc8-6d4480584c09 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:45.562376 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2d8f9f69-b833-466e-bdc8-6d4480584c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:45.610103 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f5aed-bea8-4486-97e6-9b8bf8ab25fc None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:45.610295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f5aed-bea8-4486-97e6-9b8bf8ab25fc None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:45.610564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f5aed-bea8-4486-97e6-9b8bf8ab25fc None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:45.611925 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 501/1009] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:45.617071 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d97dd5-8e36-47e6-8528-c610486dd206 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:45.617357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d97dd5-8e36-47e6-8528-c610486dd206 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:45.617636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d97dd5-8e36-47e6-8528-c610486dd206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:45.635055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-18d97dd5-8e36-47e6-8528-c610486dd206 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:45.636232 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-18d97dd5-8e36-47e6-8528-c610486dd206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:45.640433 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 509/1010] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:45 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:45.661417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f9f69-b833-466e-bdc8-6d4480584c09 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:45.676449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:45.707771 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 502/1011] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:45 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:45.730065 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d97dd5-8e36-47e6-8528-c610486dd206 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:45.776697 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f9f69-b833-466e-bdc8-6d4480584c09 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:45.782105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:45.813045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb82485-cdcf-48da-ba43-250e37c4a734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:45.813859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb82485-cdcf-48da-ba43-250e37c4a734 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:45.814444 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb82485-cdcf-48da-ba43-250e37c4a734 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:45.814727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb82485-cdcf-48da-ba43-250e37c4a734 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:45.815537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb82485-cdcf-48da-ba43-250e37c4a734 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:45.817442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb82485-cdcf-48da-ba43-250e37c4a734 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-ecb82485-cdcf-48da-ba43-250e37c4a734 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:45.829780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:45.830800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:45.830800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:45.830800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:45.830800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:45.830800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:45.830800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:45.830800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:45.830800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:45.830800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:45.830800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:45.830800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:45.830800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:45.830800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:45.830800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:45.830800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:45.830800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:45.831640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:45.831640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:45.831640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:45.831640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:45.831640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:45.831640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:45.831640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:45.831640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:45.831640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:45.831640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:45.831640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:45.831640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:45.831640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:45.831640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:45.831640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 510/1012] 10.222.0.43 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:45.837433 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb82485-cdcf-48da-ba43-250e37c4a734 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:45.840457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d97dd5-8e36-47e6-8528-c610486dd206 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:45.876021 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5c35d-b89b-4bc9-b07a-0a9013ff000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:45.877010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5c35d-b89b-4bc9-b07a-0a9013ff000c None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:45.877480 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5c35d-b89b-4bc9-b07a-0a9013ff000c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:45.877956 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5c35d-b89b-4bc9-b07a-0a9013ff000c None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:45.879384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5c35d-b89b-4bc9-b07a-0a9013ff000c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:45.880984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5c35d-b89b-4bc9-b07a-0a9013ff000c None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:45.897800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 503/1013] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jul 20 19:11:45 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:11:45.919467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:45.946984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb82485-cdcf-48da-ba43-250e37c4a734 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:45.991328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c8053-d960-45f1-b450-33e8513350d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:45.992005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c8053-d960-45f1-b450-33e8513350d5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:45.992313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c8053-d960-45f1-b450-33e8513350d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:45.992590 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c8053-d960-45f1-b450-33e8513350d5 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:45.993303 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c8053-d960-45f1-b450-33e8513350d5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:45.993608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c8053-d960-45f1-b450-33e8513350d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:45.994983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c8053-d960-45f1-b450-33e8513350d5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:46.000609 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 511/1014] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:46.006921 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5c35d-b89b-4bc9-b07a-0a9013ff000c None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:46.112138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5c35d-b89b-4bc9-b07a-0a9013ff000c None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:46.122519 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:46.149997 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016f9c5-0158-437e-b41b-3bbb49d4f383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:46.150592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016f9c5-0158-437e-b41b-3bbb49d4f383 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:46.150878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016f9c5-0158-437e-b41b-3bbb49d4f383 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:46.151155 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016f9c5-0158-437e-b41b-3bbb49d4f383 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:46.151864 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016f9c5-0158-437e-b41b-3bbb49d4f383 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:46.152175 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016f9c5-0158-437e-b41b-3bbb49d4f383 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:46.153622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016f9c5-0158-437e-b41b-3bbb49d4f383 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:46.158786 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 504/1015] 10.222.0.43 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:46.164348 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c8053-d960-45f1-b450-33e8513350d5 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:46.266698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c8053-d960-45f1-b450-33e8513350d5 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:46.299903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28987a0c-d08e-48a8-a9be-f7a8d6f778f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:46.301109 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28987a0c-d08e-48a8-a9be-f7a8d6f778f2 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:46.301391 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28987a0c-d08e-48a8-a9be-f7a8d6f778f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:46.301663 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28987a0c-d08e-48a8-a9be-f7a8d6f778f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:46.302408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28987a0c-d08e-48a8-a9be-f7a8d6f778f2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:46.303765 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28987a0c-d08e-48a8-a9be-f7a8d6f778f2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:46.308221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:46.323452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 512/1016] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jul 20 19:11:46 2022] POST /identity/v3/endpoints => generated 325 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:46.441144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69bde7-26a6-4658-be4e-48f8a28bea8d None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:46.441455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69bde7-26a6-4658-be4e-48f8a28bea8d None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:46.441664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69bde7-26a6-4658-be4e-48f8a28bea8d None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:46.443231 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 505/1017] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:46.448812 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a298290-5251-42c8-a947-7d1269341fd3 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:46.449104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a298290-5251-42c8-a947-7d1269341fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:46.449370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a298290-5251-42c8-a947-7d1269341fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:46.466964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2a298290-5251-42c8-a947-7d1269341fd3 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:46.468170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2a298290-5251-42c8-a947-7d1269341fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:46.545640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 513/1018] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:46.569763 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69bde7-26a6-4658-be4e-48f8a28bea8d None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:46.674649 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69bde7-26a6-4658-be4e-48f8a28bea8d None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:46.711027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb058b-64dd-430f-b57b-8b2f6ee281f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:46.711756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb058b-64dd-430f-b57b-8b2f6ee281f4 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:46.712034 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb058b-64dd-430f-b57b-8b2f6ee281f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:46.712327 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb058b-64dd-430f-b57b-8b2f6ee281f4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:46.722352 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb058b-64dd-430f-b57b-8b2f6ee281f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:46.731381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-38eb058b-64dd-430f-b57b-8b2f6ee281f4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:46.731381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:46.731381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:46.731381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:46.731381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:46.731381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:46.731381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:46.731381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:46.731381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:46.731381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:46.731381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:46.731381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:46.731381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:46.731381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:46.731381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:46.731381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:46.732570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:46.732570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:46.732570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:46.732570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:46.732570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:46.732570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:46.732570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:46.732570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:46.732570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:46.732570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:46.732570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:46.732570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:46.732570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:46.732570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:46.732570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:46.732570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:46.733533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:46.733533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:46.733533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:46.733533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:46.733533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:46.733533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:46.733533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:46.733533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:46.733533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:46.733533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:46.733533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:46.733533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:46.736366 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 506/1019] 10.222.0.43 () {56 vars in 1120 bytes} [Wed Jul 20 19:11:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:46.739429 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a298290-5251-42c8-a947-7d1269341fd3 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:46.838506 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a298290-5251-42c8-a947-7d1269341fd3 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:46.861583 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:46.884500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b91aa-e3ce-454d-b3ab-f3be6d41f4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:46.885134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b91aa-e3ce-454d-b3ab-f3be6d41f4db None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:46.885416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b91aa-e3ce-454d-b3ab-f3be6d41f4db None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:46.886666 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b91aa-e3ce-454d-b3ab-f3be6d41f4db None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:46.886666 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b91aa-e3ce-454d-b3ab-f3be6d41f4db None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:46.887447 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b91aa-e3ce-454d-b3ab-f3be6d41f4db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:46.888107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b91aa-e3ce-454d-b3ab-f3be6d41f4db None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:46.903619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 514/1020] 10.222.0.43 () {56 vars in 1115 bytes} [Wed Jul 20 19:11:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:46.905534 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb058b-64dd-430f-b57b-8b2f6ee281f4 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:46.938227 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:47.012412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb058b-64dd-430f-b57b-8b2f6ee281f4 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:47.048409 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca455c8-397d-42d8-9e06-22501763eccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:47.049226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca455c8-397d-42d8-9e06-22501763eccd None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:47.049629 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca455c8-397d-42d8-9e06-22501763eccd None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:47.049786 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca455c8-397d-42d8-9e06-22501763eccd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:47.061489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca455c8-397d-42d8-9e06-22501763eccd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:47.063047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca455c8-397d-42d8-9e06-22501763eccd None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:47.080585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-4ca455c8-397d-42d8-9e06-22501763eccd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:47.080585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:47.080585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:47.080585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:47.080585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:47.080585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:47.080585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:47.080585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:47.080585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:47.080585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:47.080585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:47.080585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:47.080585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:47.080585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:47.080585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:47.081514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:47.083158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:47.083158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:47.083158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:47.083158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:47.084252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 507/1021] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:47.088234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b91aa-e3ce-454d-b3ab-f3be6d41f4db None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:47.207000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b91aa-e3ce-454d-b3ab-f3be6d41f4db None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:47.244032 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a73b7-0f21-407d-9c73-3a4ccb7abe33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:47.244716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a73b7-0f21-407d-9c73-3a4ccb7abe33 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:47.245000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a73b7-0f21-407d-9c73-3a4ccb7abe33 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:47.245321 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a73b7-0f21-407d-9c73-3a4ccb7abe33 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:47.246006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a73b7-0f21-407d-9c73-3a4ccb7abe33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:47.246313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a73b7-0f21-407d-9c73-3a4ccb7abe33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:47.247829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a73b7-0f21-407d-9c73-3a4ccb7abe33 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:47.262754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 515/1022] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:47.268347 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca455c8-397d-42d8-9e06-22501763eccd None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:47.369363 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:47.386757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca455c8-397d-42d8-9e06-22501763eccd None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:47.421485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2979f5-cbde-4e4d-88a1-bc0c33f58370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:47.422278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2979f5-cbde-4e4d-88a1-bc0c33f58370 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:47.422559 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2979f5-cbde-4e4d-88a1-bc0c33f58370 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:47.422829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2979f5-cbde-4e4d-88a1-bc0c33f58370 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:47.430480 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2979f5-cbde-4e4d-88a1-bc0c33f58370 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:47.431902 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2979f5-cbde-4e4d-88a1-bc0c33f58370 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:47.446987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-8a2979f5-cbde-4e4d-88a1-bc0c33f58370 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:47.446987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:47.446987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:47.446987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:47.446987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:47.446987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:47.446987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:47.446987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:47.446987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:47.446987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:47.446987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:47.446987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:47.446987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:47.446987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:47.446987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:47.446987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:47.447976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:47.453961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:47.453961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:47.453961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:47.453961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:47.453961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:47.453961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:47.453961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:47.453961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:47.453961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:47.453961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:47.453961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 508/1023] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jul 20 19:11:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:47.456008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a73b7-0f21-407d-9c73-3a4ccb7abe33 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:47.506584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:47.562703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a73b7-0f21-407d-9c73-3a4ccb7abe33 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:47.596582 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43626e2-8b86-423c-b6db-3b47a22fdd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:47.597254 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43626e2-8b86-423c-b6db-3b47a22fdd64 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:47.597564 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43626e2-8b86-423c-b6db-3b47a22fdd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:47.597837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43626e2-8b86-423c-b6db-3b47a22fdd64 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:47.598668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43626e2-8b86-423c-b6db-3b47a22fdd64 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:47.598975 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43626e2-8b86-423c-b6db-3b47a22fdd64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:47.600420 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43626e2-8b86-423c-b6db-3b47a22fdd64 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:47.618014 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 516/1024] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jul 20 19:11:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:47.623618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2979f5-cbde-4e4d-88a1-bc0c33f58370 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:47.723718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2979f5-cbde-4e4d-88a1-bc0c33f58370 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:47.757261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a212365-8ad4-48a3-99f1-d02dede7abdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:47.757911 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a212365-8ad4-48a3-99f1-d02dede7abdc None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:47.758192 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a212365-8ad4-48a3-99f1-d02dede7abdc None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:47.758463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a212365-8ad4-48a3-99f1-d02dede7abdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:47.772875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:47.825898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a212365-8ad4-48a3-99f1-d02dede7abdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:47.827277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a212365-8ad4-48a3-99f1-d02dede7abdc None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:47.929634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 509/1025] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:47 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:11:47.931258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43626e2-8b86-423c-b6db-3b47a22fdd64 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:47.973794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b05c6b-4d90-4674-85aa-0f992489450b None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:47.974140 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b05c6b-4d90-4674-85aa-0f992489450b None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:47.974366 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b05c6b-4d90-4674-85aa-0f992489450b None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:47.975869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 510/1026] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:47.980538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc64273-9742-4180-8379-107f1edc1824 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:47.980829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc64273-9742-4180-8379-107f1edc1824 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:47.981109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc64273-9742-4180-8379-107f1edc1824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:47.997390 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-0bc64273-9742-4180-8379-107f1edc1824 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:47.998574 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0bc64273-9742-4180-8379-107f1edc1824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:48.035532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43626e2-8b86-423c-b6db-3b47a22fdd64 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:48.037317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:48.046600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:48.073628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0189b-1ecc-488f-b744-7e890440b722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:48.074425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0189b-1ecc-488f-b744-7e890440b722 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:48.074706 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0189b-1ecc-488f-b744-7e890440b722 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:48.074985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0189b-1ecc-488f-b744-7e890440b722 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:48.082879 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 511/1027] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:48.088261 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0189b-1ecc-488f-b744-7e890440b722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:48.089747 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0189b-1ecc-488f-b744-7e890440b722 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:48.103795 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc64273-9742-4180-8379-107f1edc1824 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:48.107544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-ebb0189b-1ecc-488f-b744-7e890440b722 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:48.107544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:48.107544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:48.107544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:48.107544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:48.107544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:48.107544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:48.107544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:48.107544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:48.107544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:48.107544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:48.107544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:48.107544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:48.107544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:48.107544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:48.108655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:48.109778 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:48.109778 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:48.109778 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:48.109778 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:48.113337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 517/1028] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:48.114142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0189b-1ecc-488f-b744-7e890440b722 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:48.217124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc64273-9742-4180-8379-107f1edc1824 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:48.223819 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0189b-1ecc-488f-b744-7e890440b722 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:48.251822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7a4a0-9de4-4b0e-a593-bec3da71baad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:48.253163 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7a4a0-9de4-4b0e-a593-bec3da71baad None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:48.254234 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7a4a0-9de4-4b0e-a593-bec3da71baad None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:48.254652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7a4a0-9de4-4b0e-a593-bec3da71baad None admin] PATH_INFO: `/v3/services/image` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:48.255984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7a4a0-9de4-4b0e-a593-bec3da71baad None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:48.257610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7a4a0-9de4-4b0e-a593-bec3da71baad None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:48.259646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc309e1-6edc-4a14-9241-bad3edbef52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:48.260327 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc309e1-6edc-4a14-9241-bad3edbef52b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:48.260620 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc309e1-6edc-4a14-9241-bad3edbef52b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:48.260899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc309e1-6edc-4a14-9241-bad3edbef52b None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:48.261632 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc309e1-6edc-4a14-9241-bad3edbef52b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:48.261943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc309e1-6edc-4a14-9241-bad3edbef52b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:48.263403 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc309e1-6edc-4a14-9241-bad3edbef52b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-86a7a4a0-9de4-4b0e-a593-bec3da71baad None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:48.273605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:48.274830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:48.274830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:48.274830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:48.274830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:48.274830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:48.274830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:48.274830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:48.274830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:48.274830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:48.274830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:48.274830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:48.274830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:48.274830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:48.274830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:48.274830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:48.274830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:48.275846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:48.275846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:48.275846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:48.275846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:48.275846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:48.275846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:48.275846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:48.275846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:48.275846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:48.275846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:48.275846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:48.275846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:48.275846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:48.275846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:48.278334 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 512/1029] 10.222.0.43 () {56 vars in 1105 bytes} [Wed Jul 20 19:11:48 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:48.280336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 518/1030] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:48.284370 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7a4a0-9de4-4b0e-a593-bec3da71baad None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:48.284601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc309e1-6edc-4a14-9241-bad3edbef52b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:48.303835 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:48.329984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:48.400324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7a4a0-9de4-4b0e-a593-bec3da71baad None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:48.404125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc309e1-6edc-4a14-9241-bad3edbef52b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:48.435744 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d7d3d-0c79-48d8-acc9-228b5076249e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:48.436580 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d7d3d-0c79-48d8-acc9-228b5076249e None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:48.436869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d7d3d-0c79-48d8-acc9-228b5076249e None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:48.437147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d7d3d-0c79-48d8-acc9-228b5076249e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:48.439946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25896621-a073-48f4-971e-f8db398253a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:48.440631 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25896621-a073-48f4-971e-f8db398253a4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:48.440915 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25896621-a073-48f4-971e-f8db398253a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:48.441254 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25896621-a073-48f4-971e-f8db398253a4 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:48.442452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25896621-a073-48f4-971e-f8db398253a4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:48.442786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25896621-a073-48f4-971e-f8db398253a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:48.444147 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25896621-a073-48f4-971e-f8db398253a4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:48.444560 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d7d3d-0c79-48d8-acc9-228b5076249e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:48.445974 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d7d3d-0c79-48d8-acc9-228b5076249e None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:48.454031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 519/1031] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:48.454031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558533a-c22b-4c13-b84e-3551c7243ab7 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:48.454031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558533a-c22b-4c13-b84e-3551c7243ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:48.454031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558533a-c22b-4c13-b84e-3551c7243ab7 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:48.454031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 520/1032] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:48.454798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558533a-c22b-4c13-b84e-3551c7243ab7 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:48.460425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-ba5d7d3d-0c79-48d8-acc9-228b5076249e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:48.460425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:48.460425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:48.460425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:48.460425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:48.460425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:48.460425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:48.460425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:48.460425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:48.460425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:48.460425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:48.460425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:48.460425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:48.460425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:48.460425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:48.461294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:48.462183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:48.462183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:48.462183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:48.462183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:48.462183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:48.462183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:48.462183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:48.462183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:48.462183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 513/1033] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:48.463131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd332311-ea4b-4cce-bcfb-a4db291ed5b0 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:48.463131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd332311-ea4b-4cce-bcfb-a4db291ed5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:48.463375 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd332311-ea4b-4cce-bcfb-a4db291ed5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:48.479661 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-bd332311-ea4b-4cce-bcfb-a4db291ed5b0 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:48.480890 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-bd332311-ea4b-4cce-bcfb-a4db291ed5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:48.534752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:48.547198 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 514/1034] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:48.548073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd332311-ea4b-4cce-bcfb-a4db291ed5b0 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:48.562497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558533a-c22b-4c13-b84e-3551c7243ab7 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:48.578822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:48.597501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e2f05-2498-40e2-9909-9b85d647d2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:48.598167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e2f05-2498-40e2-9909-9b85d647d2ce None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:48.598450 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e2f05-2498-40e2-9909-9b85d647d2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:48.598730 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e2f05-2498-40e2-9909-9b85d647d2ce None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:48.599437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e2f05-2498-40e2-9909-9b85d647d2ce None admin] RBAC: Adding query filter params (type=image) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:48.599748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e2f05-2498-40e2-9909-9b85d647d2ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:48.601203 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e2f05-2498-40e2-9909-9b85d647d2ce None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:48.606092 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 521/1035] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:48.606680 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e2f05-2498-40e2-9909-9b85d647d2ce None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:48.654352 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd332311-ea4b-4cce-bcfb-a4db291ed5b0 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:48.688158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453238a-2f6f-493b-962e-4f0b23cf561e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:48.688838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453238a-2f6f-493b-962e-4f0b23cf561e None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:48.689120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453238a-2f6f-493b-962e-4f0b23cf561e None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:48.689395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453238a-2f6f-493b-962e-4f0b23cf561e None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:48.690179 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453238a-2f6f-493b-962e-4f0b23cf561e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:48.690487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453238a-2f6f-493b-962e-4f0b23cf561e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:48.691946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453238a-2f6f-493b-962e-4f0b23cf561e None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:48.703379 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 515/1036] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:48.704172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453238a-2f6f-493b-962e-4f0b23cf561e None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:48.710393 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e2f05-2498-40e2-9909-9b85d647d2ce None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:48.744252 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c8f22-21b0-495a-8d84-da65eba6601d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:48.745074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c8f22-21b0-495a-8d84-da65eba6601d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:48.745385 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c8f22-21b0-495a-8d84-da65eba6601d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:48.745663 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c8f22-21b0-495a-8d84-da65eba6601d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:48.747081 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c8f22-21b0-495a-8d84-da65eba6601d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:48.748227 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c8f22-21b0-495a-8d84-da65eba6601d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:48.750044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 522/1037] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:48.750716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c8f22-21b0-495a-8d84-da65eba6601d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:48.764135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:48.794099 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:48.808083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453238a-2f6f-493b-962e-4f0b23cf561e None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:48.842274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b9bed-51be-4add-ad4a-b952a99fcf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:48.842933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b9bed-51be-4add-ad4a-b952a99fcf93 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:48.843216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b9bed-51be-4add-ad4a-b952a99fcf93 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:48.843517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b9bed-51be-4add-ad4a-b952a99fcf93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:48.844325 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b9bed-51be-4add-ad4a-b952a99fcf93 None admin] RBAC: Adding query filter params (interface=public, service_id=1f6734b262a840b799cc25a3a9964b2b, region_id=RegionOne) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:48.844642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b9bed-51be-4add-ad4a-b952a99fcf93 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:48.845999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b9bed-51be-4add-ad4a-b952a99fcf93 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:48.851537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 516/1038] 10.222.0.43 () {56 vars in 1251 bytes} [Wed Jul 20 19:11:48 2022] GET /identity/v3/endpoints?service_id=1f6734b262a840b799cc25a3a9964b2b&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:48.852229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b9bed-51be-4add-ad4a-b952a99fcf93 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:48.854516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c8f22-21b0-495a-8d84-da65eba6601d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:48.888512 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7203a8-2a81-4b2e-a54d-369405d68ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:48.889236 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7203a8-2a81-4b2e-a54d-369405d68ada None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:48.889430 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7203a8-2a81-4b2e-a54d-369405d68ada None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:48.889699 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7203a8-2a81-4b2e-a54d-369405d68ada None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:48.890524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7203a8-2a81-4b2e-a54d-369405d68ada None admin] RBAC: Adding query filter params (role.id=35471af587764a54ba1c01a3ab111054, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=267b178236e046e4bc694b2da6f86eb0) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:48.890830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7203a8-2a81-4b2e-a54d-369405d68ada None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:48.892245 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7203a8-2a81-4b2e-a54d-369405d68ada None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:48.897621 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 523/1039] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:48 2022] GET /identity/v3/role_assignments?user.id=267b178236e046e4bc694b2da6f86eb0&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=35471af587764a54ba1c01a3ab111054 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:48.898385 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7203a8-2a81-4b2e-a54d-369405d68ada None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:48.982882 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:48.984524 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b9bed-51be-4add-ad4a-b952a99fcf93 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:49.011420 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7203a8-2a81-4b2e-a54d-369405d68ada None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:49.025930 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99f5ed-8083-4a2f-9192-3da302182362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:49.026604 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99f5ed-8083-4a2f-9192-3da302182362 None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:49.026891 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99f5ed-8083-4a2f-9192-3da302182362 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:49.027169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99f5ed-8083-4a2f-9192-3da302182362 None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:49.027897 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99f5ed-8083-4a2f-9192-3da302182362 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:49.029392 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99f5ed-8083-4a2f-9192-3da302182362 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:49.054263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dff6b8-d67a-41ed-96b5-eff6e23bc95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:49.055359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dff6b8-d67a-41ed-96b5-eff6e23bc95c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:49.055359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dff6b8-d67a-41ed-96b5-eff6e23bc95c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:49.055555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dff6b8-d67a-41ed-96b5-eff6e23bc95c None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:49.056136 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dff6b8-d67a-41ed-96b5-eff6e23bc95c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:49.057908 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dff6b8-d67a-41ed-96b5-eff6e23bc95c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:49.059331 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 517/1040] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jul 20 19:11:48 2022] POST /identity/v3/projects => generated 311 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:49.071469 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 524/1041] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jul 20 19:11:48 2022] GET /identity/v3/services => generated 1957 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:11:50.911029 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fdc1c-2cf5-44be-b318-f3bc08080844 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:50.911554 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fdc1c-2cf5-44be-b318-f3bc08080844 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:50.911554 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fdc1c-2cf5-44be-b318-f3bc08080844 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:50.913017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 518/1042] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:50.918027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d03354-9ef2-40b8-a475-77c6c5c83956 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:50.918549 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d03354-9ef2-40b8-a475-77c6c5c83956 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:50.918972 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d03354-9ef2-40b8-a475-77c6c5c83956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:50.935792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-82d03354-9ef2-40b8-a475-77c6c5c83956 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:50.937019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-82d03354-9ef2-40b8-a475-77c6c5c83956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:51.012326 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 525/1043] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:50 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:51.031950 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fdc1c-2cf5-44be-b318-f3bc08080844 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:51.036327 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:51.096270 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c6754-d91a-47ce-9952-0cdcd8ff15f4 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:51.096581 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c6754-d91a-47ce-9952-0cdcd8ff15f4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:51.096850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c6754-d91a-47ce-9952-0cdcd8ff15f4 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:51.100640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 526/1044] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:51.103295 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebbcbd-f08c-448b-9700-72ea6d33ef61 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:51.103590 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebbcbd-f08c-448b-9700-72ea6d33ef61 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:51.103869 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebbcbd-f08c-448b-9700-72ea6d33ef61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:51.121364 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f1ebbcbd-f08c-448b-9700-72ea6d33ef61 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:51.122557 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f1ebbcbd-f08c-448b-9700-72ea6d33ef61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:51.151205 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fdc1c-2cf5-44be-b318-f3bc08080844 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:51.184829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e5422-fcf0-4caf-8b64-37b93bcd82cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:51.185272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:51.185551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e5422-fcf0-4caf-8b64-37b93bcd82cb None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:51.185833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e5422-fcf0-4caf-8b64-37b93bcd82cb None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:51.186106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e5422-fcf0-4caf-8b64-37b93bcd82cb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:51.192046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 527/1045] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:51.193373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0b225-8b41-4c3f-9d04-c0684deab01d None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:51.193677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0b225-8b41-4c3f-9d04-c0684deab01d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:51.193936 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0b225-8b41-4c3f-9d04-c0684deab01d None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:51.195343 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 528/1046] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:51.197174 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e5422-fcf0-4caf-8b64-37b93bcd82cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:51.200411 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b160a-fee4-4469-aa9c-29a2bf69751b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:51.200701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b160a-fee4-4469-aa9c-29a2bf69751b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:51.200988 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b160a-fee4-4469-aa9c-29a2bf69751b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:51.205383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-388e5422-fcf0-4caf-8b64-37b93bcd82cb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:51.205383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:51.205383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:51.205383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:51.205383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:51.205383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:51.205383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:51.205383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:51.205383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:51.205383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:51.205383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:51.205383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:51.205383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:51.205383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:51.205383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:51.205383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:51.206309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:51.206309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:51.206309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:51.206309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:51.206309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:51.206309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:51.206309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:51.206309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:51.206309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:51.206309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:51.206309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:51.206309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:51.206309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:51.206309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:51.206309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:51.206309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:51.209393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:51.209393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:51.209393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:51.209393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:51.209393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:51.209393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:51.209393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:51.209393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:51.209393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:51.209393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:51.209393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:51.209393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:51.209393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 519/1047] 10.222.0.43 () {56 vars in 1120 bytes} [Wed Jul 20 19:11:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:51.212965 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e5422-fcf0-4caf-8b64-37b93bcd82cb None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:51.218158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2b6b160a-fee4-4469-aa9c-29a2bf69751b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:51.219358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2b6b160a-fee4-4469-aa9c-29a2bf69751b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:51.274956 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:51.287152 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 529/1048] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:51.290137 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b160a-fee4-4469-aa9c-29a2bf69751b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:51.318397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e5422-fcf0-4caf-8b64-37b93bcd82cb None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:51.352772 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7174296-b86a-4a3d-8004-e2d449c1b407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:51.353450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7174296-b86a-4a3d-8004-e2d449c1b407 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:51.353955 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7174296-b86a-4a3d-8004-e2d449c1b407 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:51.354061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7174296-b86a-4a3d-8004-e2d449c1b407 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:51.354768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7174296-b86a-4a3d-8004-e2d449c1b407 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:51.355078 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7174296-b86a-4a3d-8004-e2d449c1b407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:51.356465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7174296-b86a-4a3d-8004-e2d449c1b407 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:51.365077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 520/1049] 10.222.0.43 () {56 vars in 1115 bytes} [Wed Jul 20 19:11:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:51.365895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7174296-b86a-4a3d-8004-e2d449c1b407 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:51.393243 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b160a-fee4-4469-aa9c-29a2bf69751b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:51.418743 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:51.428283 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403fb35-103e-4a26-825b-fe215ef3ce3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:51.429097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403fb35-103e-4a26-825b-fe215ef3ce3d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:51.429383 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403fb35-103e-4a26-825b-fe215ef3ce3d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:51.429661 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403fb35-103e-4a26-825b-fe215ef3ce3d None admin] PATH_INFO: `/v3/services/image` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:51.430318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403fb35-103e-4a26-825b-fe215ef3ce3d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:51.431682 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403fb35-103e-4a26-825b-fe215ef3ce3d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-5403fb35-103e-4a26-825b-fe215ef3ce3d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:51.444023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:51.445244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:51.445244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:51.445244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:51.445244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:51.445244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:51.445244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:51.445244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:51.445244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:51.445244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:51.445244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:51.445244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:51.445244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:51.445244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:51.445244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:51.445244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:51.445244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:51.446079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:51.446079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:51.446079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:51.446079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:51.446079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:51.446079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:51.446079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:51.446079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:51.446079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:51.446079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:51.446079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:51.446079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:51.446079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:51.446079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:51.446079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 530/1050] 10.222.0.43 () {56 vars in 1105 bytes} [Wed Jul 20 19:11:51 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:51.446868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403fb35-103e-4a26-825b-fe215ef3ce3d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:51.466556 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7174296-b86a-4a3d-8004-e2d449c1b407 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:51.491393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:51.499734 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51f2aa-6bbb-4b89-a8e5-eb80cccf88f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:51.500555 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51f2aa-6bbb-4b89-a8e5-eb80cccf88f0 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:51.500837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51f2aa-6bbb-4b89-a8e5-eb80cccf88f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:51.501108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51f2aa-6bbb-4b89-a8e5-eb80cccf88f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:51.502271 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51f2aa-6bbb-4b89-a8e5-eb80cccf88f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:51.504233 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51f2aa-6bbb-4b89-a8e5-eb80cccf88f0 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:51.505884 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 521/1051] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:51.508466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51f2aa-6bbb-4b89-a8e5-eb80cccf88f0 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:51.549808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403fb35-103e-4a26-825b-fe215ef3ce3d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:51.583772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60c163-cf49-435e-bae5-326039a0ed25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:51.584642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60c163-cf49-435e-bae5-326039a0ed25 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:51.584931 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60c163-cf49-435e-bae5-326039a0ed25 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:51.585234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60c163-cf49-435e-bae5-326039a0ed25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:51.595524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60c163-cf49-435e-bae5-326039a0ed25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:51.597004 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60c163-cf49-435e-bae5-326039a0ed25 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:51.607660 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51f2aa-6bbb-4b89-a8e5-eb80cccf88f0 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:51.614166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-dc60c163-cf49-435e-bae5-326039a0ed25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:51.614166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:51.614166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:51.614166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:51.614166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:51.614166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:51.614166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:51.614166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:51.614166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:51.614166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:51.614166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:51.614166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:51.614166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:51.614166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:51.614166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:51.615225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:51.616130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:51.616130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:51.616130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:51.616130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:51.616130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 531/1052] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:51.616639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60c163-cf49-435e-bae5-326039a0ed25 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:51.646157 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5218a21-9ca3-4171-99a8-d1ead84f3ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:51.646819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5218a21-9ca3-4171-99a8-d1ead84f3ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:51.647103 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5218a21-9ca3-4171-99a8-d1ead84f3ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:51.647378 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5218a21-9ca3-4171-99a8-d1ead84f3ab9 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:51.648082 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5218a21-9ca3-4171-99a8-d1ead84f3ab9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:51.648420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5218a21-9ca3-4171-99a8-d1ead84f3ab9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:51.649764 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5218a21-9ca3-4171-99a8-d1ead84f3ab9 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:51.654500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 522/1053] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:51.655267 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5218a21-9ca3-4171-99a8-d1ead84f3ab9 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:51.664657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:51.713927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:51.719029 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60c163-cf49-435e-bae5-326039a0ed25 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:51.753238 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17465ed1-424b-4b76-947d-91a36aa4f2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:51.753905 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17465ed1-424b-4b76-947d-91a36aa4f2d8 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:51.754194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17465ed1-424b-4b76-947d-91a36aa4f2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:51.754475 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17465ed1-424b-4b76-947d-91a36aa4f2d8 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:51.755052 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5218a21-9ca3-4171-99a8-d1ead84f3ab9 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:51.755198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17465ed1-424b-4b76-947d-91a36aa4f2d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:51.756731 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17465ed1-424b-4b76-947d-91a36aa4f2d8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:51.775276 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 532/1054] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:11:51 2022] POST /identity/v3/users => generated 285 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:51.776079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17465ed1-424b-4b76-947d-91a36aa4f2d8 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:51.795534 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633153f6-f203-4673-856b-04125b65273f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:51.796270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633153f6-f203-4673-856b-04125b65273f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:51.796707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633153f6-f203-4673-856b-04125b65273f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:51.796979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633153f6-f203-4673-856b-04125b65273f None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:51.797841 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633153f6-f203-4673-856b-04125b65273f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:51.798147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633153f6-f203-4673-856b-04125b65273f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:51.799605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633153f6-f203-4673-856b-04125b65273f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:51.814769 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 523/1055] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:51.820566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633153f6-f203-4673-856b-04125b65273f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:51.897182 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:51.901121 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17465ed1-424b-4b76-947d-91a36aa4f2d8 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:51.937874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633153f6-f203-4673-856b-04125b65273f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:51.943819 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08255d-ba3f-4f00-a0c6-2e337cdc1cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:51.945164 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08255d-ba3f-4f00-a0c6-2e337cdc1cea None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:51.945164 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08255d-ba3f-4f00-a0c6-2e337cdc1cea None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:51.945164 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08255d-ba3f-4f00-a0c6-2e337cdc1cea None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:51.946771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08255d-ba3f-4f00-a0c6-2e337cdc1cea None admin] RBAC: Adding query filter params (type=image) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:51.946771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08255d-ba3f-4f00-a0c6-2e337cdc1cea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:51.947420 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08255d-ba3f-4f00-a0c6-2e337cdc1cea None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:51.953174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 533/1056] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:51.957943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:51.959380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08255d-ba3f-4f00-a0c6-2e337cdc1cea None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:51.980589 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2116417-efb0-4998-8e79-b8257931ee98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:51.981325 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2116417-efb0-4998-8e79-b8257931ee98 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:51.981612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2116417-efb0-4998-8e79-b8257931ee98 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:51.981888 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2116417-efb0-4998-8e79-b8257931ee98 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:51.990960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2116417-efb0-4998-8e79-b8257931ee98 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:51.992436 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2116417-efb0-4998-8e79-b8257931ee98 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:52.007291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-c2116417-efb0-4998-8e79-b8257931ee98 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:52.007291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:52.007291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:52.007291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:52.007291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:52.007291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:52.007291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:52.007291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:52.007291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:52.007291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:52.007291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:52.007291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:52.007291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:52.007291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:52.007291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:52.007291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:52.008266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:52.014014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:52.014014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:52.014014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:52.014014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:52.014014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:52.014014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:52.014014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:52.014014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:52.014014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:52.014014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:52.014014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 524/1057] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jul 20 19:11:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:52.015905 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2116417-efb0-4998-8e79-b8257931ee98 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:52.073235 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08255d-ba3f-4f00-a0c6-2e337cdc1cea None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:52.109688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9194c74-f212-4dde-858b-8a6c763a0733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:52.110357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194c74-f212-4dde-858b-8a6c763a0733 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:52.110644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194c74-f212-4dde-858b-8a6c763a0733 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:52.110921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194c74-f212-4dde-858b-8a6c763a0733 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:52.111665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9194c74-f212-4dde-858b-8a6c763a0733 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:52.113095 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9194c74-f212-4dde-858b-8a6c763a0733 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:52.122860 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2116417-efb0-4998-8e79-b8257931ee98 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:52.132401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 534/1058] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jul 20 19:11:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:11:52.178396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e096216-ff58-460f-a4be-9182cef50e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:52.179217 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e096216-ff58-460f-a4be-9182cef50e7f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:52.179586 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e096216-ff58-460f-a4be-9182cef50e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:52.179922 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e096216-ff58-460f-a4be-9182cef50e7f None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:52.180922 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e096216-ff58-460f-a4be-9182cef50e7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:52.181341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e096216-ff58-460f-a4be-9182cef50e7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:52.183087 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e096216-ff58-460f-a4be-9182cef50e7f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:52.208467 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 525/1059] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jul 20 19:11:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:52.210081 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9194c74-f212-4dde-858b-8a6c763a0733 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:52.236572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:52.329260 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9194c74-f212-4dde-858b-8a6c763a0733 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:52.373636 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d606c-fb92-434a-ade3-9dc9fa19428d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:52.373636 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d606c-fb92-434a-ade3-9dc9fa19428d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:52.373921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d606c-fb92-434a-ade3-9dc9fa19428d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:52.373996 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d606c-fb92-434a-ade3-9dc9fa19428d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:52.398422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d606c-fb92-434a-ade3-9dc9fa19428d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:52.401062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d606c-fb92-434a-ade3-9dc9fa19428d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:52.426458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-576d606c-fb92-434a-ade3-9dc9fa19428d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:52.426458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:52.426458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:52.426458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:52.426458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:52.426458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:52.426458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:52.426458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:52.426458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:52.426458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:52.426458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:52.426458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:52.426458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:52.426458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:52.426458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:52.427425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:52.428628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:52.428628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:52.428628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:52.428628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:52.428628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 535/1060] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:52.434075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e096216-ff58-460f-a4be-9182cef50e7f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:52.470927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:52.558672 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e096216-ff58-460f-a4be-9182cef50e7f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:52.597112 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffc36c-2126-4128-8726-0d4d42e36e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:52.598085 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffc36c-2126-4128-8726-0d4d42e36e0c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:52.598374 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffc36c-2126-4128-8726-0d4d42e36e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:52.598653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffc36c-2126-4128-8726-0d4d42e36e0c None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:52.599400 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffc36c-2126-4128-8726-0d4d42e36e0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:52.599710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffc36c-2126-4128-8726-0d4d42e36e0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:52.601123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffc36c-2126-4128-8726-0d4d42e36e0c None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:52.621129 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 526/1061] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:52.622294 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d606c-fb92-434a-ade3-9dc9fa19428d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:52.714582 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:52.754728 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d606c-fb92-434a-ade3-9dc9fa19428d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:52.791577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f58eb-86dd-4bfc-be80-a5d86aceab89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:52.792402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f58eb-86dd-4bfc-be80-a5d86aceab89 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:52.792688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f58eb-86dd-4bfc-be80-a5d86aceab89 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:52.792963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f58eb-86dd-4bfc-be80-a5d86aceab89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:52.801121 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f58eb-86dd-4bfc-be80-a5d86aceab89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:52.802141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f58eb-86dd-4bfc-be80-a5d86aceab89 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:52.818127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-424f58eb-86dd-4bfc-be80-a5d86aceab89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:52.818127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:52.818127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:52.818127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:52.818127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:52.818127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:52.818127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:52.818127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:52.818127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:52.818127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:52.818127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:52.818127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:52.818127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:52.818127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:52.818127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:52.819034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:52.819917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:52.819917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:52.819917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:52.819917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:52.819917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:52.819917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:52.819917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:52.819917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:52.819917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 536/1062] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:52.825082 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffc36c-2126-4128-8726-0d4d42e36e0c None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:52.922917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:52.933972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffc36c-2126-4128-8726-0d4d42e36e0c None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:52.970984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b156b0-780e-4474-bec0-2b450fc806ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:52.971681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b156b0-780e-4474-bec0-2b450fc806ca None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:52.971941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b156b0-780e-4474-bec0-2b450fc806ca None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:52.972398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b156b0-780e-4474-bec0-2b450fc806ca None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:52.973326 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b156b0-780e-4474-bec0-2b450fc806ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:52.973657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b156b0-780e-4474-bec0-2b450fc806ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:52.975110 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b156b0-780e-4474-bec0-2b450fc806ca None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:52.991619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 527/1063] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:52.994567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f58eb-86dd-4bfc-be80-a5d86aceab89 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:53.101005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f58eb-86dd-4bfc-be80-a5d86aceab89 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:53.130705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:53.136778 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625ed06-c9d9-4184-99ed-86c2738125a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:53.137671 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625ed06-c9d9-4184-99ed-86c2738125a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:53.137953 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625ed06-c9d9-4184-99ed-86c2738125a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:53.138228 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625ed06-c9d9-4184-99ed-86c2738125a4 None admin] PATH_INFO: `/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/267b178236e046e4bc694b2da6f86eb0/roles/35471af587764a54ba1c01a3ab111054` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:53.140465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625ed06-c9d9-4184-99ed-86c2738125a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e2a50ebbb48417c911df499d98a0157, user_id=267b178236e046e4bc694b2da6f86eb0, role_id=35471af587764a54ba1c01a3ab111054)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:53.141898 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625ed06-c9d9-4184-99ed-86c2738125a4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:53.154646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 537/1064] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:11:52 2022] PUT /identity/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/267b178236e046e4bc694b2da6f86eb0/roles/35471af587764a54ba1c01a3ab111054 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:11:53.885247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccc491-2213-44a2-a9f4-02a9d7c2972a None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:53.885657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccc491-2213-44a2-a9f4-02a9d7c2972a None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:53.885838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccc491-2213-44a2-a9f4-02a9d7c2972a None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:53.887420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 528/1065] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:53.892510 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126bb1f-2dc8-43d3-a825-cd1bc8b49355 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:53.892807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126bb1f-2dc8-43d3-a825-cd1bc8b49355 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:53.893078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126bb1f-2dc8-43d3-a825-cd1bc8b49355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:53.910502 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5126bb1f-2dc8-43d3-a825-cd1bc8b49355 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:53.911703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5126bb1f-2dc8-43d3-a825-cd1bc8b49355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:53.981491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 538/1066] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:54.004837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fccc491-2213-44a2-a9f4-02a9d7c2972a None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:54.108605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fccc491-2213-44a2-a9f4-02a9d7c2972a None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:54.142526 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb98e1a-b4dc-4020-8ec3-27d6c428e2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:54.143238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb98e1a-b4dc-4020-8ec3-27d6c428e2ad None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:54.143522 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb98e1a-b4dc-4020-8ec3-27d6c428e2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:54.143800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb98e1a-b4dc-4020-8ec3-27d6c428e2ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:54.154528 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb98e1a-b4dc-4020-8ec3-27d6c428e2ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:54.162349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-fcb98e1a-b4dc-4020-8ec3-27d6c428e2ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:54.162349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:54.162349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:54.162349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:54.162349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:54.162349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:54.162349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:54.162349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:54.162349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:54.162349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:54.162349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:54.162349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:54.162349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:54.162349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:54.162349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:54.162349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:54.163266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:54.163266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:54.163266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:54.163266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:54.163266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:54.163266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:54.163266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:54.163266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:54.163266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:54.163266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:54.163266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:54.163266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:54.163266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:54.163266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:54.163266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:54.163266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:54.164107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:54.164107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:54.164107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:54.164107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:54.164107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:54.164107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:54.164107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:54.164107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:54.164107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:54.164107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:54.164107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:54.164107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:54.172036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 529/1067] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:54.172172 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126bb1f-2dc8-43d3-a825-cd1bc8b49355 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:54.223466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e08def-f67c-4d5b-99a0-4533a5b7d10f None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:54.223800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e08def-f67c-4d5b-99a0-4533a5b7d10f None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:54.224028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e08def-f67c-4d5b-99a0-4533a5b7d10f None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:54.225591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 530/1068] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:54.230324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d80064-330c-48d1-b38b-3f15d23c7aa6 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:54.230616 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d80064-330c-48d1-b38b-3f15d23c7aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:54.230999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d80064-330c-48d1-b38b-3f15d23c7aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:54.249439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-42d80064-330c-48d1-b38b-3f15d23c7aa6 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:54.250685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-42d80064-330c-48d1-b38b-3f15d23c7aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:54.259560 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:54.280029 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126bb1f-2dc8-43d3-a825-cd1bc8b49355 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:54.304229 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:54.318676 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aef5a2-7e06-41ef-a8bc-93c389a15c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:54.319344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aef5a2-7e06-41ef-a8bc-93c389a15c08 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:54.319627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aef5a2-7e06-41ef-a8bc-93c389a15c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:54.319901 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aef5a2-7e06-41ef-a8bc-93c389a15c08 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:54.320817 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aef5a2-7e06-41ef-a8bc-93c389a15c08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:54.321234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aef5a2-7e06-41ef-a8bc-93c389a15c08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:54.322602 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aef5a2-7e06-41ef-a8bc-93c389a15c08 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:54.326218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 531/1069] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:54.335481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 539/1070] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:54.337015 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d80064-330c-48d1-b38b-3f15d23c7aa6 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:54.350548 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aef5a2-7e06-41ef-a8bc-93c389a15c08 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:54.458904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d80064-330c-48d1-b38b-3f15d23c7aa6 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:54.466679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aef5a2-7e06-41ef-a8bc-93c389a15c08 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:54.510849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00073ace-c7aa-42ac-8588-80deb0632693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:54.511066 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1caffac-8f38-4974-926a-7bc9cc1294d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:54.511501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00073ace-c7aa-42ac-8588-80deb0632693 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:54.511785 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00073ace-c7aa-42ac-8588-80deb0632693 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:54.511884 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caffac-8f38-4974-926a-7bc9cc1294d9 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:54.512060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00073ace-c7aa-42ac-8588-80deb0632693 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:54.512159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caffac-8f38-4974-926a-7bc9cc1294d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:54.512406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caffac-8f38-4974-926a-7bc9cc1294d9 None admin] PATH_INFO: `/v3/users/7e09e12138444094a0e3ff2aeac05b23` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:54.527217 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:54.532160 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1caffac-8f38-4974-926a-7bc9cc1294d9 None admin] RBAC: Authorizing `identity:get_user(user_id=7e09e12138444094a0e3ff2aeac05b23)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:54.533734 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1caffac-8f38-4974-926a-7bc9cc1294d9 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:54.541284 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 532/1071] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:11:54 2022] GET /identity/v3/users/7e09e12138444094a0e3ff2aeac05b23 => generated 285 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:54.542530 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1caffac-8f38-4974-926a-7bc9cc1294d9 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:54.579937 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:54.597182 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00073ace-c7aa-42ac-8588-80deb0632693 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:54.598725 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00073ace-c7aa-42ac-8588-80deb0632693 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:54.667248 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1caffac-8f38-4974-926a-7bc9cc1294d9 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:54.703921 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a2178-0a4c-4b52-b7b5-87c8fd94e2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:54.704740 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a2178-0a4c-4b52-b7b5-87c8fd94e2cb None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:54.705031 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a2178-0a4c-4b52-b7b5-87c8fd94e2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:54.705307 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a2178-0a4c-4b52-b7b5-87c8fd94e2cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:54.712111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 540/1072] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 19:11:54.712932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00073ace-c7aa-42ac-8588-80deb0632693 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:54.775514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:54.778196 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a2178-0a4c-4b52-b7b5-87c8fd94e2cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:54.779657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a2178-0a4c-4b52-b7b5-87c8fd94e2cb None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:54.804225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:54.829881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00073ace-c7aa-42ac-8588-80deb0632693 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:54.871964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f9601-9d31-4921-b4f5-f1e9e5e744fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:54.872794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f9601-9d31-4921-b4f5-f1e9e5e744fb None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:54.873079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f9601-9d31-4921-b4f5-f1e9e5e744fb None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:54.873356 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f9601-9d31-4921-b4f5-f1e9e5e744fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:54.885825 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f9601-9d31-4921-b4f5-f1e9e5e744fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:54.887383 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f9601-9d31-4921-b4f5-f1e9e5e744fb None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:54.907698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-864f9601-9d31-4921-b4f5-f1e9e5e744fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:54.907698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:54.907698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:54.907698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:54.907698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:54.907698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:54.907698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:54.907698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:54.907698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:54.907698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:54.907698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:54.907698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:54.907698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:54.907698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:54.907698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:54.908740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:54.910130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:54.910130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:54.910130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:54.910130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:54.910130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 541/1073] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:54.915653 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f9601-9d31-4921-b4f5-f1e9e5e744fb None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:54.927436 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 533/1074] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 19:11:54.933385 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a2178-0a4c-4b52-b7b5-87c8fd94e2cb None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:55.047896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:55.050493 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f9601-9d31-4921-b4f5-f1e9e5e744fb None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:55.051261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a2178-0a4c-4b52-b7b5-87c8fd94e2cb None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:55.092251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff29c7-48e4-413b-adc4-269f42dcc975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:55.092931 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff29c7-48e4-413b-adc4-269f42dcc975 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:55.093224 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff29c7-48e4-413b-adc4-269f42dcc975 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:55.093507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff29c7-48e4-413b-adc4-269f42dcc975 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:55.094249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff29c7-48e4-413b-adc4-269f42dcc975 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:55.094563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff29c7-48e4-413b-adc4-269f42dcc975 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:55.095955 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff29c7-48e4-413b-adc4-269f42dcc975 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:55.096915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cea51-a451-4a1d-a566-b172e3285ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:55.098497 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cea51-a451-4a1d-a566-b172e3285ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:55.098925 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cea51-a451-4a1d-a566-b172e3285ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:55.099281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:55.099540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cea51-a451-4a1d-a566-b172e3285ba1 None admin] PATH_INFO: `/v3/projects/faf98d284ea447a498dc75090f2002df` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:55.101381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740cea51-a451-4a1d-a566-b172e3285ba1 None admin] RBAC: Authorizing `identity:get_project(project_id=faf98d284ea447a498dc75090f2002df)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:55.102825 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740cea51-a451-4a1d-a566-b172e3285ba1 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:55.109825 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 534/1075] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:11:54 2022] GET /identity/v3/projects/faf98d284ea447a498dc75090f2002df => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:55.120330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 542/1076] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:55.120637 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cea51-a451-4a1d-a566-b172e3285ba1 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:55.190945 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b9585-5cae-453f-8cf4-08a16479792e None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:55.191488 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b9585-5cae-453f-8cf4-08a16479792e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:55.191488 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b9585-5cae-453f-8cf4-08a16479792e None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:55.193284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 543/1077] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:55.199541 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36d481-c5cf-44de-bcb5-776c0942ef15 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:55.199873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36d481-c5cf-44de-bcb5-776c0942ef15 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:55.200098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36d481-c5cf-44de-bcb5-776c0942ef15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:55.224049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-bb36d481-c5cf-44de-bcb5-776c0942ef15 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:55.224049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bb36d481-c5cf-44de-bcb5-776c0942ef15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:55.237417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cea51-a451-4a1d-a566-b172e3285ba1 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:55.280074 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59950459-6b97-4d19-bfff-c96385c96937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:55.280767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59950459-6b97-4d19-bfff-c96385c96937 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:55.281046 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59950459-6b97-4d19-bfff-c96385c96937 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:55.281309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59950459-6b97-4d19-bfff-c96385c96937 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:55.282149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59950459-6b97-4d19-bfff-c96385c96937 None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=faf98d284ea447a498dc75090f2002df, user.id=7e09e12138444094a0e3ff2aeac05b23) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:55.283077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59950459-6b97-4d19-bfff-c96385c96937 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:55.283876 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59950459-6b97-4d19-bfff-c96385c96937 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:55.291988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 535/1078] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:55 2022] GET /identity/v3/role_assignments?user.id=7e09e12138444094a0e3ff2aeac05b23&scope.project.id=faf98d284ea447a498dc75090f2002df&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:55.299619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 544/1079] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:55.323849 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59950459-6b97-4d19-bfff-c96385c96937 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:55.362107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:55.443597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59950459-6b97-4d19-bfff-c96385c96937 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:55.478372 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561ee35-a23e-414a-b900-2087d6497b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:55.479188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561ee35-a23e-414a-b900-2087d6497b59 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:55.479471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561ee35-a23e-414a-b900-2087d6497b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:55.479749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561ee35-a23e-414a-b900-2087d6497b59 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:55.489817 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561ee35-a23e-414a-b900-2087d6497b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:55.498159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-9561ee35-a23e-414a-b900-2087d6497b59 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:55.498159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:55.498159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:55.498159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:55.498159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:55.498159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:55.498159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:55.498159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:55.498159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:55.498159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:55.498159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:55.498159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:55.498159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:55.498159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:55.498159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:55.498159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:55.499238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:55.499238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:55.499238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:55.499238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:55.499238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:55.499238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:55.499238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:55.499238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:55.499238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:55.499238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:55.499238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:55.499238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:55.499238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:55.499238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:55.499238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:55.499238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:55.500101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:55.500101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:55.500101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:55.500101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:55.500101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:55.500101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:55.500101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:55.500101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:55.500101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:55.500101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:55.500101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:55.500101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:55.500101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 536/1080] 10.222.0.43 () {56 vars in 1120 bytes} [Wed Jul 20 19:11:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:55.505861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36d481-c5cf-44de-bcb5-776c0942ef15 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:55.602456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36d481-c5cf-44de-bcb5-776c0942ef15 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:55.627143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:55.636759 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684086e6-fdc8-4536-84b4-161155b43d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:55.637419 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684086e6-fdc8-4536-84b4-161155b43d1b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:55.637697 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684086e6-fdc8-4536-84b4-161155b43d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:55.637967 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684086e6-fdc8-4536-84b4-161155b43d1b None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:55.638708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684086e6-fdc8-4536-84b4-161155b43d1b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:55.639011 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684086e6-fdc8-4536-84b4-161155b43d1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:55.640402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684086e6-fdc8-4536-84b4-161155b43d1b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:55.649043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 545/1081] 10.222.0.43 () {56 vars in 1115 bytes} [Wed Jul 20 19:11:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:55.654662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561ee35-a23e-414a-b900-2087d6497b59 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:55.743113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:55.752565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561ee35-a23e-414a-b900-2087d6497b59 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:55.786416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd36383-f073-4a17-b013-37cfe01fe65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:55.787202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd36383-f073-4a17-b013-37cfe01fe65e None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:55.787483 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd36383-f073-4a17-b013-37cfe01fe65e None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:55.787754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd36383-f073-4a17-b013-37cfe01fe65e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:55.797979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd36383-f073-4a17-b013-37cfe01fe65e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:55.799405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd36383-f073-4a17-b013-37cfe01fe65e None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:55.817311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-ebd36383-f073-4a17-b013-37cfe01fe65e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:55.817311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:55.817311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:55.817311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:55.817311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:55.817311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:55.817311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:55.817311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:55.817311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:55.817311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:55.817311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:55.817311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:55.817311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:55.817311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:55.817311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:55.818188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:55.819077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:55.819077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:55.819077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:55.819077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:55.819077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 537/1082] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:55.824186 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684086e6-fdc8-4536-84b4-161155b43d1b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:55.920847 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684086e6-fdc8-4536-84b4-161155b43d1b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:55.957124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1c2ca-d485-4905-a01e-294936db58fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:55.957781 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1c2ca-d485-4905-a01e-294936db58fc None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:55.958062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1c2ca-d485-4905-a01e-294936db58fc None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:55.958333 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1c2ca-d485-4905-a01e-294936db58fc None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:55.959052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1c2ca-d485-4905-a01e-294936db58fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:55.959382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1c2ca-d485-4905-a01e-294936db58fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:55.960769 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1c2ca-d485-4905-a01e-294936db58fc None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:55.972538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 546/1083] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:55.978120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd36383-f073-4a17-b013-37cfe01fe65e None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:56.075468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd36383-f073-4a17-b013-37cfe01fe65e None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:56.107162 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe744d-6075-4142-bdcf-b1ebcb8db8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:56.107941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe744d-6075-4142-bdcf-b1ebcb8db8fb None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:56.108219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe744d-6075-4142-bdcf-b1ebcb8db8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:56.108517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe744d-6075-4142-bdcf-b1ebcb8db8fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:56.116136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe744d-6075-4142-bdcf-b1ebcb8db8fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:56.117581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe744d-6075-4142-bdcf-b1ebcb8db8fb None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:56.131581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-bfbe744d-6075-4142-bdcf-b1ebcb8db8fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:56.131581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:56.131581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:56.131581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:56.131581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:56.131581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:56.131581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:56.131581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:56.131581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:56.131581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:56.131581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:56.131581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:56.131581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:56.131581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:56.131581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:56.131581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:56.132544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:56.133513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:56.133513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:56.133513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:56.133513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:56.133513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:56.133513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:56.133513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:56.133513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:56.133513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:56.133513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:11:56.133513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 538/1084] 10.222.0.43 () {56 vars in 1093 bytes} [Wed Jul 20 19:11:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:56.138669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1c2ca-d485-4905-a01e-294936db58fc None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:56.186928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:56.235087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1c2ca-d485-4905-a01e-294936db58fc None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:56.267488 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc725a-83ed-4dc0-942c-881b591d5991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:56.268143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc725a-83ed-4dc0-942c-881b591d5991 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:56.268459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc725a-83ed-4dc0-942c-881b591d5991 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:56.268733 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc725a-83ed-4dc0-942c-881b591d5991 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:56.269597 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc725a-83ed-4dc0-942c-881b591d5991 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:56.269900 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc725a-83ed-4dc0-942c-881b591d5991 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:56.271309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc725a-83ed-4dc0-942c-881b591d5991 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:56.288111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 547/1085] 10.222.0.43 () {56 vars in 1133 bytes} [Wed Jul 20 19:11:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:56.293851 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe744d-6075-4142-bdcf-b1ebcb8db8fb None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:56.297570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:56.399619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe744d-6075-4142-bdcf-b1ebcb8db8fb None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:56.433783 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081858c-55b6-4d5e-b8d1-f6d00c3ddca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:56.434427 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081858c-55b6-4d5e-b8d1-f6d00c3ddca7 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:56.434712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081858c-55b6-4d5e-b8d1-f6d00c3ddca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:56.434988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081858c-55b6-4d5e-b8d1-f6d00c3ddca7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:56.510642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081858c-55b6-4d5e-b8d1-f6d00c3ddca7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:56.513400 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081858c-55b6-4d5e-b8d1-f6d00c3ddca7 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:56.514893 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:56.612226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 539/1086] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:11:56.617974 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc725a-83ed-4dc0-942c-881b591d5991 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:56.720966 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc725a-83ed-4dc0-942c-881b591d5991 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:56.729606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:56.757101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90034cf-8379-4092-afd0-c4ccf4de110e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:56.757900 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90034cf-8379-4092-afd0-c4ccf4de110e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:56.758189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90034cf-8379-4092-afd0-c4ccf4de110e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:56.758459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90034cf-8379-4092-afd0-c4ccf4de110e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:56.768941 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90034cf-8379-4092-afd0-c4ccf4de110e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:56.770854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90034cf-8379-4092-afd0-c4ccf4de110e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:56.789456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-c90034cf-8379-4092-afd0-c4ccf4de110e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:56.789456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:56.789456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:56.789456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:56.789456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:56.789456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:56.789456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:56.789456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:56.789456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:56.789456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:56.789456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:56.789456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:56.789456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:56.789456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:56.789456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:56.790408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:56.791292 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:56.791292 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:56.791292 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:56.791292 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:56.791292 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 548/1087] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:56.797149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081858c-55b6-4d5e-b8d1-f6d00c3ddca7 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:56.898111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081858c-55b6-4d5e-b8d1-f6d00c3ddca7 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:56.900503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:56.931962 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b6b90-ebd8-45c2-80d4-c89a96a18634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:56.932910 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b6b90-ebd8-45c2-80d4-c89a96a18634 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:56.933196 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b6b90-ebd8-45c2-80d4-c89a96a18634 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:56.933476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b6b90-ebd8-45c2-80d4-c89a96a18634 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:56.934206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b6b90-ebd8-45c2-80d4-c89a96a18634 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:56.934516 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b6b90-ebd8-45c2-80d4-c89a96a18634 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:56.935876 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b6b90-ebd8-45c2-80d4-c89a96a18634 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:56.947867 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 540/1088] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:56.954873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90034cf-8379-4092-afd0-c4ccf4de110e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:57.064834 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90034cf-8379-4092-afd0-c4ccf4de110e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:57.100937 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbc484-753c-4a6f-a9b9-664b44788e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:57.101732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbc484-753c-4a6f-a9b9-664b44788e58 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:57.102019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbc484-753c-4a6f-a9b9-664b44788e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:57.102303 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbc484-753c-4a6f-a9b9-664b44788e58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:57.118122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbc484-753c-4a6f-a9b9-664b44788e58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:57.120124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbc484-753c-4a6f-a9b9-664b44788e58 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:57.135436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-33cbc484-753c-4a6f-a9b9-664b44788e58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:57.135436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:57.135436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:57.135436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:57.135436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:57.135436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:57.135436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:57.135436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:57.135436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:57.135436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:57.135436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:57.135436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:57.135436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:57.135436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:57.135436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:57.136476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:57.137368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:57.137368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:57.137368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:57.137368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:57.137368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:57.137368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:57.137368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:57.137368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:57.137368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 549/1089] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:57.142588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b6b90-ebd8-45c2-80d4-c89a96a18634 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:57.259544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b6b90-ebd8-45c2-80d4-c89a96a18634 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:57.295723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa06ca4-cae7-433a-9ae4-6b03a029653b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:57.296432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa06ca4-cae7-433a-9ae4-6b03a029653b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:57.296717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa06ca4-cae7-433a-9ae4-6b03a029653b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:57.296991 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa06ca4-cae7-433a-9ae4-6b03a029653b None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:57.297797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa06ca4-cae7-433a-9ae4-6b03a029653b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:57.298110 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa06ca4-cae7-433a-9ae4-6b03a029653b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:57.299582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa06ca4-cae7-433a-9ae4-6b03a029653b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:57.308496 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3c786-1373-401e-8e21-8d2b6563ec92 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:57.309105 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3c786-1373-401e-8e21-8d2b6563ec92 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:57.309214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3c786-1373-401e-8e21-8d2b6563ec92 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:57.310814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 550/1090] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:11:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:57.313147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 541/1091] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:57.319194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10372db-92b6-41e2-bd88-583b49815ed3 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:57.319516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10372db-92b6-41e2-bd88-583b49815ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:57.320091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10372db-92b6-41e2-bd88-583b49815ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:57.321646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa06ca4-cae7-433a-9ae4-6b03a029653b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:57.338530 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a10372db-92b6-41e2-bd88-583b49815ed3 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:57.339723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a10372db-92b6-41e2-bd88-583b49815ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:57.351972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:57.358734 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:57.421904 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 551/1092] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:57.445839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10372db-92b6-41e2-bd88-583b49815ed3 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:57.457317 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa06ca4-cae7-433a-9ae4-6b03a029653b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:57.499544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774ddd7-32aa-4056-ad76-2cf372034f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:57.500533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774ddd7-32aa-4056-ad76-2cf372034f5e None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:57.500827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774ddd7-32aa-4056-ad76-2cf372034f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:57.501138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774ddd7-32aa-4056-ad76-2cf372034f5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:57.502547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774ddd7-32aa-4056-ad76-2cf372034f5e None admin] RBAC: Adding query filter params (role.id=35471af587764a54ba1c01a3ab111054, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=267b178236e046e4bc694b2da6f86eb0) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:57.502906 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774ddd7-32aa-4056-ad76-2cf372034f5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:57.505340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774ddd7-32aa-4056-ad76-2cf372034f5e None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:57.511552 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 542/1093] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:57 2022] GET /identity/v3/role_assignments?user.id=267b178236e046e4bc694b2da6f86eb0&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=35471af587764a54ba1c01a3ab111054 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:57.560085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10372db-92b6-41e2-bd88-583b49815ed3 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:57.599390 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ecfa6-665d-4dbe-b17a-eb5e1cf940f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:57.600112 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ecfa6-665d-4dbe-b17a-eb5e1cf940f6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:57.600432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ecfa6-665d-4dbe-b17a-eb5e1cf940f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:57.600709 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ecfa6-665d-4dbe-b17a-eb5e1cf940f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:57.612037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ecfa6-665d-4dbe-b17a-eb5e1cf940f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:57.620532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-a12ecfa6-665d-4dbe-b17a-eb5e1cf940f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:57.620532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:57.620532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:57.620532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:57.620532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:57.620532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:57.620532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:57.620532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:57.620532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:57.620532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:57.620532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:57.620532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:57.620532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:57.620532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:57.620532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:57.620532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:57.621990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:57.621990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:57.621990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:57.621990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:57.621990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:57.621990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:57.621990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:57.621990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:57.621990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:57.621990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:57.621990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:57.621990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:57.621990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:57.621990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:57.621990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:57.621990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:57.622849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:57.622849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:57.622849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:57.622849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:57.622849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:57.622849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:57.622849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:57.622849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:57.622849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:57.622849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:57.622849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:57.622849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:11:57.622849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 552/1094] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:57.628174 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774ddd7-32aa-4056-ad76-2cf372034f5e None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:57.722711 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:57.735124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774ddd7-32aa-4056-ad76-2cf372034f5e None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:57.769961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f07b13-7561-4c20-843e-ee8b7a5e8f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:57.770632 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f07b13-7561-4c20-843e-ee8b7a5e8f54 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:57.770917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f07b13-7561-4c20-843e-ee8b7a5e8f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:57.771194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f07b13-7561-4c20-843e-ee8b7a5e8f54 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:57.771946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f07b13-7561-4c20-843e-ee8b7a5e8f54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:57.772255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f07b13-7561-4c20-843e-ee8b7a5e8f54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:57.773643 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f07b13-7561-4c20-843e-ee8b7a5e8f54 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:57.783311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 543/1095] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:57.789883 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ecfa6-665d-4dbe-b17a-eb5e1cf940f6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:57.802197 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:57.895788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ecfa6-665d-4dbe-b17a-eb5e1cf940f6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:57.930270 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76f76a-2291-4bea-9f9f-01f29568efcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:57.931148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76f76a-2291-4bea-9f9f-01f29568efcf None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:57.931413 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76f76a-2291-4bea-9f9f-01f29568efcf None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:57.931683 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76f76a-2291-4bea-9f9f-01f29568efcf None admin] PATH_INFO: `/v3/users/7e09e12138444094a0e3ff2aeac05b23` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:57.933242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76f76a-2291-4bea-9f9f-01f29568efcf None admin] RBAC: Authorizing `identity:get_user(user_id=7e09e12138444094a0e3ff2aeac05b23)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:57.934711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76f76a-2291-4bea-9f9f-01f29568efcf None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:57.936410 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 553/1096] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:11:57 2022] GET /identity/v3/users/7e09e12138444094a0e3ff2aeac05b23 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:57.942956 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f07b13-7561-4c20-843e-ee8b7a5e8f54 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:58.049324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f07b13-7561-4c20-843e-ee8b7a5e8f54 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:58.091706 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ca1b2-9c19-43b0-84af-5afe701fbb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:58.092665 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ca1b2-9c19-43b0-84af-5afe701fbb13 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:58.092947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ca1b2-9c19-43b0-84af-5afe701fbb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:58.093223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ca1b2-9c19-43b0-84af-5afe701fbb13 None admin] PATH_INFO: `/v3/projects/faf98d284ea447a498dc75090f2002df` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:58.094414 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ca1b2-9c19-43b0-84af-5afe701fbb13 None admin] RBAC: Authorizing `identity:get_project(project_id=faf98d284ea447a498dc75090f2002df)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:58.095824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ca1b2-9c19-43b0-84af-5afe701fbb13 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:58.098118 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 544/1097] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:11:57 2022] GET /identity/v3/projects/faf98d284ea447a498dc75090f2002df => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:58.104518 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76f76a-2291-4bea-9f9f-01f29568efcf None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:58.185652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:58.209212 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76f76a-2291-4bea-9f9f-01f29568efcf None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:58.244397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40434b08-c538-4ec3-a503-853fc7953bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:58.245294 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40434b08-c538-4ec3-a503-853fc7953bc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:58.245577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40434b08-c538-4ec3-a503-853fc7953bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:58.245848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40434b08-c538-4ec3-a503-853fc7953bc6 None admin] PATH_INFO: `/v3/projects/faf98d284ea447a498dc75090f2002df/users/7e09e12138444094a0e3ff2aeac05b23/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:58.247819 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40434b08-c538-4ec3-a503-853fc7953bc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=faf98d284ea447a498dc75090f2002df, user_id=7e09e12138444094a0e3ff2aeac05b23, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:58.249279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40434b08-c538-4ec3-a503-853fc7953bc6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:58.262385 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 554/1098] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:11:58 2022] PUT /identity/v3/projects/faf98d284ea447a498dc75090f2002df/users/7e09e12138444094a0e3ff2aeac05b23/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:12:00.292522 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d613d-6070-477c-8245-2d5b6784a6f8 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:00.293057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d613d-6070-477c-8245-2d5b6784a6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:00.293057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d613d-6070-477c-8245-2d5b6784a6f8 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:00.294419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 545/1099] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:00.299577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baf1b0-7e22-4180-b97f-7f4b98830696 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:00.299983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baf1b0-7e22-4180-b97f-7f4b98830696 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:00.300125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baf1b0-7e22-4180-b97f-7f4b98830696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:00.317965 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f0baf1b0-7e22-4180-b97f-7f4b98830696 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:00.319141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f0baf1b0-7e22-4180-b97f-7f4b98830696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:00.386412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 555/1100] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:00.410133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d613d-6070-477c-8245-2d5b6784a6f8 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:00.429309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:00.510487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d613d-6070-477c-8245-2d5b6784a6f8 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:00.542790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb77a16-dfa1-4cfb-85e7-8ed407befe64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:00.543494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb77a16-dfa1-4cfb-85e7-8ed407befe64 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:00.543768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb77a16-dfa1-4cfb-85e7-8ed407befe64 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:00.544038 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb77a16-dfa1-4cfb-85e7-8ed407befe64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:00.554253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb77a16-dfa1-4cfb-85e7-8ed407befe64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:12:00.562039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-9eb77a16-dfa1-4cfb-85e7-8ed407befe64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:00.562039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:12:00.562039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:12:00.562039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:12:00.562039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:12:00.562039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:12:00.562039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:12:00.562039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:12:00.562039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:12:00.562039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:12:00.562039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:12:00.562039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:12:00.562039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:12:00.562039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:12:00.562039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:12:00.562039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:12:00.563218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:12:00.563218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:12:00.563218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:12:00.563218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:12:00.563218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:12:00.563218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:12:00.563218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:12:00.563218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:12:00.563218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:12:00.563218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:12:00.563218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:12:00.563218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:12:00.563218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:12:00.563218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:12:00.563218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:12:00.563218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:12:00.564476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:12:00.564476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:12:00.564476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:12:00.564476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:12:00.564476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:12:00.564476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:12:00.564476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:12:00.564476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:12:00.564476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:12:00.564476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:12:00.564476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:00.564476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:12:00.564476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 546/1101] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:12:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:12:00.569902 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0baf1b0-7e22-4180-b97f-7f4b98830696 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:00.634043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:00.668775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0baf1b0-7e22-4180-b97f-7f4b98830696 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:00.711458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a982a2-5952-497a-aa90-578e95dfb0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:00.712127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a982a2-5952-497a-aa90-578e95dfb0ea None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:00.712442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a982a2-5952-497a-aa90-578e95dfb0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:00.712720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a982a2-5952-497a-aa90-578e95dfb0ea None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:00.713474 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a982a2-5952-497a-aa90-578e95dfb0ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:00.713783 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a982a2-5952-497a-aa90-578e95dfb0ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:00.715150 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a982a2-5952-497a-aa90-578e95dfb0ea None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:00.723684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 556/1102] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:12:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:00.729570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb77a16-dfa1-4cfb-85e7-8ed407befe64 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:00.807970 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:00.829195 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb77a16-dfa1-4cfb-85e7-8ed407befe64 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:00.861788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240b240-fe3a-47d5-be34-ec65f297b2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:00.862697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240b240-fe3a-47d5-be34-ec65f297b2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:00.863487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240b240-fe3a-47d5-be34-ec65f297b2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:00.863761 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240b240-fe3a-47d5-be34-ec65f297b2e1 None admin] PATH_INFO: `/v3/users/7e09e12138444094a0e3ff2aeac05b23` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:00.864961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240b240-fe3a-47d5-be34-ec65f297b2e1 None admin] RBAC: Authorizing `identity:get_user(user_id=7e09e12138444094a0e3ff2aeac05b23)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:00.866382 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240b240-fe3a-47d5-be34-ec65f297b2e1 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:00.867999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 547/1103] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:12:00 2022] GET /identity/v3/users/7e09e12138444094a0e3ff2aeac05b23 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:00.874001 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a982a2-5952-497a-aa90-578e95dfb0ea None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:00.970963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a982a2-5952-497a-aa90-578e95dfb0ea None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:00.996417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:01.004890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5bbb01-d350-4c7a-beeb-5ffd6d115199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:01.005539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bbb01-d350-4c7a-beeb-5ffd6d115199 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:01.005825 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bbb01-d350-4c7a-beeb-5ffd6d115199 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:01.006131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bbb01-d350-4c7a-beeb-5ffd6d115199 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:01.075406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5bbb01-d350-4c7a-beeb-5ffd6d115199 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:01.076833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5bbb01-d350-4c7a-beeb-5ffd6d115199 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:01.172055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 557/1104] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:12:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:12:01.178085 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240b240-fe3a-47d5-be34-ec65f297b2e1 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:01.273885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240b240-fe3a-47d5-be34-ec65f297b2e1 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:01.306440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d3d22-6cde-44b2-8283-3def5712686c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:01.307550 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d3d22-6cde-44b2-8283-3def5712686c None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:01.307838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d3d22-6cde-44b2-8283-3def5712686c None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:01.308114 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d3d22-6cde-44b2-8283-3def5712686c None admin] PATH_INFO: `/v3/projects/faf98d284ea447a498dc75090f2002df` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:01.309334 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d3d22-6cde-44b2-8283-3def5712686c None admin] RBAC: Authorizing `identity:get_project(project_id=faf98d284ea447a498dc75090f2002df)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:01.310752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d3d22-6cde-44b2-8283-3def5712686c None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:01.312455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 548/1105] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:12:01 2022] GET /identity/v3/projects/faf98d284ea447a498dc75090f2002df => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:01.318644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5bbb01-d350-4c7a-beeb-5ffd6d115199 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:01.349253 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:01.418962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5bbb01-d350-4c7a-beeb-5ffd6d115199 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:01.452323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0bb68-54ab-4ee6-b063-a82dd3ca4605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:01.452980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0bb68-54ab-4ee6-b063-a82dd3ca4605 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:01.453266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0bb68-54ab-4ee6-b063-a82dd3ca4605 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:01.453546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0bb68-54ab-4ee6-b063-a82dd3ca4605 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:01.454380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0bb68-54ab-4ee6-b063-a82dd3ca4605 None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=faf98d284ea447a498dc75090f2002df, user.id=7e09e12138444094a0e3ff2aeac05b23) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:01.454698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0bb68-54ab-4ee6-b063-a82dd3ca4605 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:01.456148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0bb68-54ab-4ee6-b063-a82dd3ca4605 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:01.462519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 558/1106] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:12:01 2022] GET /identity/v3/role_assignments?user.id=7e09e12138444094a0e3ff2aeac05b23&scope.project.id=faf98d284ea447a498dc75090f2002df&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:03.477416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cb390-3759-47d5-8ddb-650426f263b2 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:03.477931 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cb390-3759-47d5-8ddb-650426f263b2 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:03.478020 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cb390-3759-47d5-8ddb-650426f263b2 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:03.479349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 549/1107] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:03.484568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957867f6-58e6-4e88-8330-e7618da8bb5d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:03.484959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957867f6-58e6-4e88-8330-e7618da8bb5d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:03.485124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957867f6-58e6-4e88-8330-e7618da8bb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:03.501836 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-957867f6-58e6-4e88-8330-e7618da8bb5d None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:03.503009 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-957867f6-58e6-4e88-8330-e7618da8bb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:03.568376 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 559/1108] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:03.592354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5cb390-3759-47d5-8ddb-650426f263b2 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:03.608146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:03.692055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5cb390-3759-47d5-8ddb-650426f263b2 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:03.725531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194c920-be89-4a9b-955b-3831ff34f8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:03.726309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194c920-be89-4a9b-955b-3831ff34f8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:03.726594 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194c920-be89-4a9b-955b-3831ff34f8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:03.726870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194c920-be89-4a9b-955b-3831ff34f8f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:03.728358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194c920-be89-4a9b-955b-3831ff34f8f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:03.729774 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194c920-be89-4a9b-955b-3831ff34f8f3 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:03.731351 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 550/1109] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:12:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:03.737123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957867f6-58e6-4e88-8330-e7618da8bb5d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:03.760963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:03.833321 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957867f6-58e6-4e88-8330-e7618da8bb5d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:03.869933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff4e8ed-74a0-4bf7-bbef-09e013dca203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:03.870528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4e8ed-74a0-4bf7-bbef-09e013dca203 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:03.870810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4e8ed-74a0-4bf7-bbef-09e013dca203 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:03.871081 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4e8ed-74a0-4bf7-bbef-09e013dca203 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:03.871803 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff4e8ed-74a0-4bf7-bbef-09e013dca203 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:03.873320 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff4e8ed-74a0-4bf7-bbef-09e013dca203 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:03.892463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 560/1110] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:12:03 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:12:05.931760 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3563b-d9ce-4bc4-8d96-f9617c06be60 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:05.932606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3563b-d9ce-4bc4-8d96-f9617c06be60 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:05.932606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3563b-d9ce-4bc4-8d96-f9617c06be60 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:05.933995 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 551/1111] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:05.939287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5acae-143f-4330-83d8-bd1a5409a27f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:05.939713 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5acae-143f-4330-83d8-bd1a5409a27f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:05.939843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5acae-143f-4330-83d8-bd1a5409a27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:05.956904 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7bb5acae-143f-4330-83d8-bd1a5409a27f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:05.958081 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7bb5acae-143f-4330-83d8-bd1a5409a27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:06.022405 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 561/1112] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:06.045913 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3563b-d9ce-4bc4-8d96-f9617c06be60 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:06.127533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:06.145861 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3563b-d9ce-4bc4-8d96-f9617c06be60 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:06.179016 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb18c8-e659-4364-a461-f4026172b14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:06.179859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb18c8-e659-4364-a461-f4026172b14d None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:06.180146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb18c8-e659-4364-a461-f4026172b14d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:06.180465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb18c8-e659-4364-a461-f4026172b14d None admin] PATH_INFO: `/v3/roles/abe200f23fc84ba397dd8c67874f0f2a` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:06.182012 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb18c8-e659-4364-a461-f4026172b14d None admin] RBAC: Authorizing `identity:get_role(role_id=abe200f23fc84ba397dd8c67874f0f2a)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:06.183585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb18c8-e659-4364-a461-f4026172b14d None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:06.184968 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 552/1113] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:12:06 2022] GET /identity/v3/roles/abe200f23fc84ba397dd8c67874f0f2a => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:06.190728 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb5acae-143f-4330-83d8-bd1a5409a27f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:06.200697 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:06.289759 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb5acae-143f-4330-83d8-bd1a5409a27f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:06.322348 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06745316-207c-4375-8493-8da5e83876eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:06.323206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06745316-207c-4375-8493-8da5e83876eb None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:06.323485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06745316-207c-4375-8493-8da5e83876eb None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:06.323761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06745316-207c-4375-8493-8da5e83876eb None admin] PATH_INFO: `/v3/users/38cb82fcd15c42c1b171211c59a47798` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:06.337148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06745316-207c-4375-8493-8da5e83876eb None admin] RBAC: Authorizing `identity:get_user(user_id=38cb82fcd15c42c1b171211c59a47798)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:06.338578 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06745316-207c-4375-8493-8da5e83876eb None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:06.340256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 562/1114] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:12:06 2022] GET /identity/v3/users/38cb82fcd15c42c1b171211c59a47798 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:06.345941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb18c8-e659-4364-a461-f4026172b14d None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:06.442306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb18c8-e659-4364-a461-f4026172b14d None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:06.475165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262e2d0-645f-40a0-8a8a-d21bdb8675c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:06.475800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262e2d0-645f-40a0-8a8a-d21bdb8675c1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:06.476077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262e2d0-645f-40a0-8a8a-d21bdb8675c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:06.476373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262e2d0-645f-40a0-8a8a-d21bdb8675c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:06.493553 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:06.543594 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262e2d0-645f-40a0-8a8a-d21bdb8675c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:06.545126 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262e2d0-645f-40a0-8a8a-d21bdb8675c1 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:06.654809 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 553/1115] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:12:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:12:06.660849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06745316-207c-4375-8493-8da5e83876eb None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:06.730896 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:06.767452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06745316-207c-4375-8493-8da5e83876eb None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:06.801428 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1072470-2f46-4e11-89a5-24b6adef31b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:06.802289 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1072470-2f46-4e11-89a5-24b6adef31b6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:06.802571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1072470-2f46-4e11-89a5-24b6adef31b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:06.802846 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1072470-2f46-4e11-89a5-24b6adef31b6 None admin] PATH_INFO: `/v3/projects/faf98d284ea447a498dc75090f2002df` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:06.804019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1072470-2f46-4e11-89a5-24b6adef31b6 None admin] RBAC: Authorizing `identity:get_project(project_id=faf98d284ea447a498dc75090f2002df)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:06.805480 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1072470-2f46-4e11-89a5-24b6adef31b6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:06.807178 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 563/1116] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:12:06 2022] GET /identity/v3/projects/faf98d284ea447a498dc75090f2002df => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:06.813618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262e2d0-645f-40a0-8a8a-d21bdb8675c1 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:06.861889 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:06.910237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262e2d0-645f-40a0-8a8a-d21bdb8675c1 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:06.943150 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412594de-e576-421e-981f-db3ded65d252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:06.943793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412594de-e576-421e-981f-db3ded65d252 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:06.944083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412594de-e576-421e-981f-db3ded65d252 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:06.944386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412594de-e576-421e-981f-db3ded65d252 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:06.945225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412594de-e576-421e-981f-db3ded65d252 None admin] RBAC: Adding query filter params (role.id=abe200f23fc84ba397dd8c67874f0f2a, scope.project.id=faf98d284ea447a498dc75090f2002df, user.id=38cb82fcd15c42c1b171211c59a47798) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:06.945537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412594de-e576-421e-981f-db3ded65d252 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:06.946954 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412594de-e576-421e-981f-db3ded65d252 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:06.952255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 554/1117] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:12:06 2022] GET /identity/v3/role_assignments?user.id=38cb82fcd15c42c1b171211c59a47798&scope.project.id=faf98d284ea447a498dc75090f2002df&role.id=abe200f23fc84ba397dd8c67874f0f2a => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:08.915480 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d22ea-dc75-4f8f-a000-9eff12b3cfa7 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:08.916071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d22ea-dc75-4f8f-a000-9eff12b3cfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:08.916071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d22ea-dc75-4f8f-a000-9eff12b3cfa7 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:08.917478 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 564/1118] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:08.923463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f1366-3563-4bc9-915a-ef51e4ab3d7a None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:08.924012 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f1366-3563-4bc9-915a-ef51e4ab3d7a None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:08.924012 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f1366-3563-4bc9-915a-ef51e4ab3d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:08.941487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-9a1f1366-3563-4bc9-915a-ef51e4ab3d7a None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:08.942669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-9a1f1366-3563-4bc9-915a-ef51e4ab3d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:09.006470 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 555/1119] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:09.029394 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d22ea-dc75-4f8f-a000-9eff12b3cfa7 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:09.133151 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d22ea-dc75-4f8f-a000-9eff12b3cfa7 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:09.171728 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f14d00-8502-4490-b5e8-d54aa999ab9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:09.172538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f14d00-8502-4490-b5e8-d54aa999ab9f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:09.172825 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f14d00-8502-4490-b5e8-d54aa999ab9f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:09.173103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f14d00-8502-4490-b5e8-d54aa999ab9f None admin] PATH_INFO: `/v3/roles/abe200f23fc84ba397dd8c67874f0f2a` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:09.174601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f14d00-8502-4490-b5e8-d54aa999ab9f None admin] RBAC: Authorizing `identity:get_role(role_id=abe200f23fc84ba397dd8c67874f0f2a)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:09.176006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f14d00-8502-4490-b5e8-d54aa999ab9f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:09.177422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 565/1120] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:12:09 2022] GET /identity/v3/roles/abe200f23fc84ba397dd8c67874f0f2a => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:12:09.183755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f1366-3563-4bc9-915a-ef51e4ab3d7a None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:09.227971 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:09.279657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f1366-3563-4bc9-915a-ef51e4ab3d7a None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:09.312965 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efbf6b-0413-4c31-a637-a75bf6dec3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:09.313820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efbf6b-0413-4c31-a637-a75bf6dec3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:09.314101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efbf6b-0413-4c31-a637-a75bf6dec3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:09.314373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efbf6b-0413-4c31-a637-a75bf6dec3d6 None admin] PATH_INFO: `/v3/users/38cb82fcd15c42c1b171211c59a47798` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:09.315533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efbf6b-0413-4c31-a637-a75bf6dec3d6 None admin] RBAC: Authorizing `identity:get_user(user_id=38cb82fcd15c42c1b171211c59a47798)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:09.317505 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efbf6b-0413-4c31-a637-a75bf6dec3d6 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:09.319088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 556/1121] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:12:09 2022] GET /identity/v3/users/38cb82fcd15c42c1b171211c59a47798 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:09.324886 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f14d00-8502-4490-b5e8-d54aa999ab9f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:09.327879 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:09.429843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f14d00-8502-4490-b5e8-d54aa999ab9f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:09.465816 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d822e-3720-422a-a373-4af7bccbe7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:09.466665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d822e-3720-422a-a373-4af7bccbe7ad None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:09.466949 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d822e-3720-422a-a373-4af7bccbe7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:09.467225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d822e-3720-422a-a373-4af7bccbe7ad None admin] PATH_INFO: `/v3/projects/faf98d284ea447a498dc75090f2002df` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:09.468428 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d822e-3720-422a-a373-4af7bccbe7ad None admin] RBAC: Authorizing `identity:get_project(project_id=faf98d284ea447a498dc75090f2002df)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:09.469847 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d822e-3720-422a-a373-4af7bccbe7ad None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:09.471529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 566/1122] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:12:09 2022] GET /identity/v3/projects/faf98d284ea447a498dc75090f2002df => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:09.477698 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efbf6b-0413-4c31-a637-a75bf6dec3d6 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:09.576264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efbf6b-0413-4c31-a637-a75bf6dec3d6 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:09.589493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:09.610572 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5cb1a-94ae-425e-82ed-0514c8bec7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:09.611459 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5cb1a-94ae-425e-82ed-0514c8bec7f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:09.611741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5cb1a-94ae-425e-82ed-0514c8bec7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:09.612016 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5cb1a-94ae-425e-82ed-0514c8bec7f2 None admin] PATH_INFO: `/v3/projects/faf98d284ea447a498dc75090f2002df/users/38cb82fcd15c42c1b171211c59a47798/roles/abe200f23fc84ba397dd8c67874f0f2a` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:09.614387 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5cb1a-94ae-425e-82ed-0514c8bec7f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=faf98d284ea447a498dc75090f2002df, user_id=38cb82fcd15c42c1b171211c59a47798, role_id=abe200f23fc84ba397dd8c67874f0f2a)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:09.615812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5cb1a-94ae-425e-82ed-0514c8bec7f2 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:09.630444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 557/1123] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:12:09 2022] PUT /identity/v3/projects/faf98d284ea447a498dc75090f2002df/users/38cb82fcd15c42c1b171211c59a47798/roles/abe200f23fc84ba397dd8c67874f0f2a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:12:11.634627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5799f-9a27-4f8e-ab87-ad2508b11b5a None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:11.635143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5799f-9a27-4f8e-ab87-ad2508b11b5a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:11.635143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5799f-9a27-4f8e-ab87-ad2508b11b5a None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:11.636521 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 567/1124] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:11.641731 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467eac6-7f35-480e-8ecb-777b75ffaf41 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:11.642147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467eac6-7f35-480e-8ecb-777b75ffaf41 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:11.642287 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467eac6-7f35-480e-8ecb-777b75ffaf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:11.658505 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-8467eac6-7f35-480e-8ecb-777b75ffaf41 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:11.659688 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-8467eac6-7f35-480e-8ecb-777b75ffaf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:11.724912 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 558/1125] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:11.748559 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5799f-9a27-4f8e-ab87-ad2508b11b5a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:11.822448 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:11.853477 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5799f-9a27-4f8e-ab87-ad2508b11b5a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:11.887354 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb052e3-ec51-4e44-8cc2-4be7c8c3b1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:11.888130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb052e3-ec51-4e44-8cc2-4be7c8c3b1ea None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:11.888443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb052e3-ec51-4e44-8cc2-4be7c8c3b1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:11.888743 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb052e3-ec51-4e44-8cc2-4be7c8c3b1ea None admin] PATH_INFO: `/v3/roles/abe200f23fc84ba397dd8c67874f0f2a` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:11.890244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb052e3-ec51-4e44-8cc2-4be7c8c3b1ea None admin] RBAC: Authorizing `identity:get_role(role_id=abe200f23fc84ba397dd8c67874f0f2a)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:11.891599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb052e3-ec51-4e44-8cc2-4be7c8c3b1ea None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:11.892989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 568/1126] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:12:11 2022] GET /identity/v3/roles/abe200f23fc84ba397dd8c67874f0f2a => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:11.898821 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467eac6-7f35-480e-8ecb-777b75ffaf41 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:11.995286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467eac6-7f35-480e-8ecb-777b75ffaf41 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:11.996771 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:12.029146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fa4fc-2241-48e7-8184-476a4345373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:12.030003 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fa4fc-2241-48e7-8184-476a4345373b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:12.030279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fa4fc-2241-48e7-8184-476a4345373b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:12.030547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fa4fc-2241-48e7-8184-476a4345373b None admin] PATH_INFO: `/v3/users/38cb82fcd15c42c1b171211c59a47798` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:12.031704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6fa4fc-2241-48e7-8184-476a4345373b None admin] RBAC: Authorizing `identity:get_user(user_id=38cb82fcd15c42c1b171211c59a47798)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:12.033273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6fa4fc-2241-48e7-8184-476a4345373b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:12.035380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 559/1127] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:12:11 2022] GET /identity/v3/users/38cb82fcd15c42c1b171211c59a47798 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:12.041064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb052e3-ec51-4e44-8cc2-4be7c8c3b1ea None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:12.135958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb052e3-ec51-4e44-8cc2-4be7c8c3b1ea None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:12.168089 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7566dc-c9e8-4eff-8844-28ffd33a4cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:12.168682 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7566dc-c9e8-4eff-8844-28ffd33a4cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:12.168962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7566dc-c9e8-4eff-8844-28ffd33a4cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:12.169239 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7566dc-c9e8-4eff-8844-28ffd33a4cb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:12.177585 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:12.234350 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7566dc-c9e8-4eff-8844-28ffd33a4cb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:12.235782 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7566dc-c9e8-4eff-8844-28ffd33a4cb1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:12.329080 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 569/1128] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:12:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:12:12.335638 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fa4fc-2241-48e7-8184-476a4345373b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:12.433623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fa4fc-2241-48e7-8184-476a4345373b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:12.466592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1136b1-7b65-4869-aafe-d77384c9980f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:12.467443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1136b1-7b65-4869-aafe-d77384c9980f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:12.467730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1136b1-7b65-4869-aafe-d77384c9980f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:12.468010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1136b1-7b65-4869-aafe-d77384c9980f None admin] PATH_INFO: `/v3/projects/faf98d284ea447a498dc75090f2002df` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:12.469214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1136b1-7b65-4869-aafe-d77384c9980f None admin] RBAC: Authorizing `identity:get_project(project_id=faf98d284ea447a498dc75090f2002df)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:12.470639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1136b1-7b65-4869-aafe-d77384c9980f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:12.472912 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 560/1129] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:12:12 2022] GET /identity/v3/projects/faf98d284ea447a498dc75090f2002df => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:12.479202 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7566dc-c9e8-4eff-8844-28ffd33a4cb1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:12.517019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:12.575475 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7566dc-c9e8-4eff-8844-28ffd33a4cb1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:12.610022 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd83fc-e87c-4dd7-8af2-864398202251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:12.610663 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd83fc-e87c-4dd7-8af2-864398202251 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:12.610947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd83fc-e87c-4dd7-8af2-864398202251 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:12.611221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd83fc-e87c-4dd7-8af2-864398202251 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:12.612048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd83fc-e87c-4dd7-8af2-864398202251 None admin] RBAC: Adding query filter params (role.id=abe200f23fc84ba397dd8c67874f0f2a, scope.project.id=faf98d284ea447a498dc75090f2002df, user.id=38cb82fcd15c42c1b171211c59a47798) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:12.612467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd83fc-e87c-4dd7-8af2-864398202251 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:12.613876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd83fc-e87c-4dd7-8af2-864398202251 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:12.619856 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 570/1130] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:12:12 2022] GET /identity/v3/role_assignments?user.id=38cb82fcd15c42c1b171211c59a47798&scope.project.id=faf98d284ea447a498dc75090f2002df&role.id=abe200f23fc84ba397dd8c67874f0f2a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:14.641006 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d5e7f-d9d5-4cbb-9184-1bfcc16f30da None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:14.641531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d5e7f-d9d5-4cbb-9184-1bfcc16f30da None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:14.641531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d5e7f-d9d5-4cbb-9184-1bfcc16f30da None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:14.642836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 561/1131] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:14.648234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98426c9-d0d9-4036-a976-b10dde586356 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:14.648694 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98426c9-d0d9-4036-a976-b10dde586356 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:14.648804 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98426c9-d0d9-4036-a976-b10dde586356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:14.665561 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a98426c9-d0d9-4036-a976-b10dde586356 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:14.666799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a98426c9-d0d9-4036-a976-b10dde586356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:14.739858 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 571/1132] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:12:14.763918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d5e7f-d9d5-4cbb-9184-1bfcc16f30da None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:14.804316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:14.863342 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d5e7f-d9d5-4cbb-9184-1bfcc16f30da None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:14.899697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47be3fc-9c6f-40d7-96c0-1bab4737c9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:14.900465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47be3fc-9c6f-40d7-96c0-1bab4737c9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:14.900972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47be3fc-9c6f-40d7-96c0-1bab4737c9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:14.901445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47be3fc-9c6f-40d7-96c0-1bab4737c9b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:14.903691 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47be3fc-9c6f-40d7-96c0-1bab4737c9b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:14.905131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47be3fc-9c6f-40d7-96c0-1bab4737c9b8 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:14.906737 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 562/1133] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:12:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:14.913016 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98426c9-d0d9-4036-a976-b10dde586356 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:14.942828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:15.008471 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98426c9-d0d9-4036-a976-b10dde586356 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:15.041109 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cdf117-fbe5-46a9-b956-bd61f8fe1306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:15.041757 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdf117-fbe5-46a9-b956-bd61f8fe1306 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:15.042037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdf117-fbe5-46a9-b956-bd61f8fe1306 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:15.042309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdf117-fbe5-46a9-b956-bd61f8fe1306 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:15.043027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cdf117-fbe5-46a9-b956-bd61f8fe1306 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:15.044470 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cdf117-fbe5-46a9-b956-bd61f8fe1306 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:15.069740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 572/1134] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jul 20 19:12:14 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:12:17.128763 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cb4b0-fc60-444b-b700-f21248a96326 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:17.129323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cb4b0-fc60-444b-b700-f21248a96326 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:17.129323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cb4b0-fc60-444b-b700-f21248a96326 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:17.130702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 563/1135] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:17.136266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0421d-8cd7-4dbd-a888-fbc286df98b6 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:17.136786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0421d-8cd7-4dbd-a888-fbc286df98b6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:17.136892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0421d-8cd7-4dbd-a888-fbc286df98b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:17.157464 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0bf0421d-8cd7-4dbd-a888-fbc286df98b6 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:17.158660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0bf0421d-8cd7-4dbd-a888-fbc286df98b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:17.230752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 573/1136] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:17.255424 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cb4b0-fc60-444b-b700-f21248a96326 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:17.358371 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cb4b0-fc60-444b-b700-f21248a96326 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:17.366377 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:17.392920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9b17e-ffcb-4d0b-8eb8-2ccce319830e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:17.394260 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9b17e-ffcb-4d0b-8eb8-2ccce319830e None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:17.394544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9b17e-ffcb-4d0b-8eb8-2ccce319830e None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:17.394817 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9b17e-ffcb-4d0b-8eb8-2ccce319830e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:17.395993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9b17e-ffcb-4d0b-8eb8-2ccce319830e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:17.397440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9b17e-ffcb-4d0b-8eb8-2ccce319830e None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:17.399040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 564/1137] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:12:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:17.404958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0421d-8cd7-4dbd-a888-fbc286df98b6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:17.420763 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:17.510849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0421d-8cd7-4dbd-a888-fbc286df98b6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:17.543679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2285c-e9fa-4327-84ae-9f63438ac3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:17.544363 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2285c-e9fa-4327-84ae-9f63438ac3f4 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:17.544648 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2285c-e9fa-4327-84ae-9f63438ac3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:17.544923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2285c-e9fa-4327-84ae-9f63438ac3f4 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:17.545640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2285c-e9fa-4327-84ae-9f63438ac3f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:17.547071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2285c-e9fa-4327-84ae-9f63438ac3f4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:17.568829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 574/1138] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:12:17 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:12:19.653182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fef86-3793-459c-a4d5-061ac5d4b2e8 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:19.653693 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fef86-3793-459c-a4d5-061ac5d4b2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:19.653693 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fef86-3793-459c-a4d5-061ac5d4b2e8 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:19.655007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 565/1139] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:19.660749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3ce7a-fd72-4239-b6d3-331e693be6a1 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:19.661180 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3ce7a-fd72-4239-b6d3-331e693be6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:19.661316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3ce7a-fd72-4239-b6d3-331e693be6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:19.677850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-32f3ce7a-fd72-4239-b6d3-331e693be6a1 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:19.679055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-32f3ce7a-fd72-4239-b6d3-331e693be6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:19.741137 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:19.747584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 575/1140] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:19.772125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fef86-3793-459c-a4d5-061ac5d4b2e8 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:19.871102 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fef86-3793-459c-a4d5-061ac5d4b2e8 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:19.904777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e5009-393e-4df8-99d5-0d743a3a222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:19.905481 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e5009-393e-4df8-99d5-0d743a3a222b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:19.905761 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e5009-393e-4df8-99d5-0d743a3a222b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:19.906034 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e5009-393e-4df8-99d5-0d743a3a222b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:19.915852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e5009-393e-4df8-99d5-0d743a3a222b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:12:19.923405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-433e5009-393e-4df8-99d5-0d743a3a222b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:19.923405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:12:19.923405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:12:19.923405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:12:19.923405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:12:19.923405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:12:19.923405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:12:19.923405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:12:19.923405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:12:19.923405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:12:19.923405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:12:19.923405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:12:19.923405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:12:19.923405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:12:19.923405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:12:19.923405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:12:19.924631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:12:19.924631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:12:19.924631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:12:19.924631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:12:19.924631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:12:19.924631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:12:19.924631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:12:19.924631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:12:19.924631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:12:19.924631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:12:19.924631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:12:19.924631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:12:19.924631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:12:19.924631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:12:19.924631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:12:19.924631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:12:19.926223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:12:19.926223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:12:19.926223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:12:19.926223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:12:19.926223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:12:19.926223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:12:19.926223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:12:19.926223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:12:19.926223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:12:19.926223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:12:19.926223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:19.926223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:12:19.926223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 566/1141] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:12:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:12:19.931242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3ce7a-fd72-4239-b6d3-331e693be6a1 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:20.031124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3ce7a-fd72-4239-b6d3-331e693be6a1 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:20.065607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c09c14-86f5-4b33-8be8-57e4242acb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:20.066775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c09c14-86f5-4b33-8be8-57e4242acb6c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:20.067056 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c09c14-86f5-4b33-8be8-57e4242acb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:20.067332 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c09c14-86f5-4b33-8be8-57e4242acb6c None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:20.068087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c09c14-86f5-4b33-8be8-57e4242acb6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:20.068426 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c09c14-86f5-4b33-8be8-57e4242acb6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:20.069802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c09c14-86f5-4b33-8be8-57e4242acb6c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:20.078510 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 576/1142] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:12:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:20.084369 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e5009-393e-4df8-99d5-0d743a3a222b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:20.118357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:20.186319 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e5009-393e-4df8-99d5-0d743a3a222b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:20.218340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3530895-b1d6-47a2-922e-183c4813203a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:20.219219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3530895-b1d6-47a2-922e-183c4813203a None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:20.219503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3530895-b1d6-47a2-922e-183c4813203a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:20.219781 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3530895-b1d6-47a2-922e-183c4813203a None admin] PATH_INFO: `/v3/users/8f2a5b7a162e433593d177f42cc6a6e4` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:20.233779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3530895-b1d6-47a2-922e-183c4813203a None admin] RBAC: Authorizing `identity:get_user(user_id=8f2a5b7a162e433593d177f42cc6a6e4)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:20.235224 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3530895-b1d6-47a2-922e-183c4813203a None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:20.236961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 567/1143] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:12:20 2022] GET /identity/v3/users/8f2a5b7a162e433593d177f42cc6a6e4 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:20.243317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c09c14-86f5-4b33-8be8-57e4242acb6c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:20.303850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:20.342300 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c09c14-86f5-4b33-8be8-57e4242acb6c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:20.376642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4681f6-faf9-4249-ad98-b920588e7e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:20.377287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4681f6-faf9-4249-ad98-b920588e7e8e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:20.377579 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4681f6-faf9-4249-ad98-b920588e7e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:20.378008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4681f6-faf9-4249-ad98-b920588e7e8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:20.443802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4681f6-faf9-4249-ad98-b920588e7e8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:20.445224 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4681f6-faf9-4249-ad98-b920588e7e8e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:20.502662 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:20.542817 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 577/1144] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:12:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:12:20.548883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3530895-b1d6-47a2-922e-183c4813203a None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:20.637030 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:20.646736 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3530895-b1d6-47a2-922e-183c4813203a None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:20.680339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966a15d-531a-4e51-b297-c9c6213b609d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:20.681197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966a15d-531a-4e51-b297-c9c6213b609d None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:20.681476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966a15d-531a-4e51-b297-c9c6213b609d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:20.681746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966a15d-531a-4e51-b297-c9c6213b609d None admin] PATH_INFO: `/v3/projects/5fc7b32810234a30a638de35a2abf536` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:20.683160 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966a15d-531a-4e51-b297-c9c6213b609d None admin] RBAC: Authorizing `identity:get_project(project_id=5fc7b32810234a30a638de35a2abf536)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:20.684593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966a15d-531a-4e51-b297-c9c6213b609d None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:20.686267 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 568/1145] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:12:20 2022] GET /identity/v3/projects/5fc7b32810234a30a638de35a2abf536 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:20.692465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4681f6-faf9-4249-ad98-b920588e7e8e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:20.794181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4681f6-faf9-4249-ad98-b920588e7e8e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:20.830096 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78659db4-8e57-4a6c-9a28-45f98c63cdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:20.830743 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78659db4-8e57-4a6c-9a28-45f98c63cdea None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:20.831553 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78659db4-8e57-4a6c-9a28-45f98c63cdea None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:20.831826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78659db4-8e57-4a6c-9a28-45f98c63cdea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:20.832686 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78659db4-8e57-4a6c-9a28-45f98c63cdea None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=5fc7b32810234a30a638de35a2abf536, user.id=8f2a5b7a162e433593d177f42cc6a6e4) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:20.832993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78659db4-8e57-4a6c-9a28-45f98c63cdea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:20.834420 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78659db4-8e57-4a6c-9a28-45f98c63cdea None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:20.840163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 578/1146] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:12:20 2022] GET /identity/v3/role_assignments?user.id=8f2a5b7a162e433593d177f42cc6a6e4&scope.project.id=5fc7b32810234a30a638de35a2abf536&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:22.823520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50859e68-97ee-434d-b3f3-dde166b9a164 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:22.824142 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50859e68-97ee-434d-b3f3-dde166b9a164 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:22.824142 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50859e68-97ee-434d-b3f3-dde166b9a164 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:22.825461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 569/1147] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:22.830760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b508d60-d41c-4c5c-b5d8-b0a7bb2ba3e2 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:22.831691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b508d60-d41c-4c5c-b5d8-b0a7bb2ba3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:22.831808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b508d60-d41c-4c5c-b5d8-b0a7bb2ba3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:22.850716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3b508d60-d41c-4c5c-b5d8-b0a7bb2ba3e2 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:22.851920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3b508d60-d41c-4c5c-b5d8-b0a7bb2ba3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:22.872040 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:22.917630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 579/1148] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:22.939994 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50859e68-97ee-434d-b3f3-dde166b9a164 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:23.037906 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50859e68-97ee-434d-b3f3-dde166b9a164 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:23.069818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a7715-993c-44cd-b9a0-49029a964b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:23.070590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a7715-993c-44cd-b9a0-49029a964b19 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:23.070868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a7715-993c-44cd-b9a0-49029a964b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:23.071139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a7715-993c-44cd-b9a0-49029a964b19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:23.081558 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a7715-993c-44cd-b9a0-49029a964b19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:12:23.089062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-4e4a7715-993c-44cd-b9a0-49029a964b19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:23.089062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:12:23.089062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:12:23.089062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:12:23.089062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:12:23.089062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:12:23.089062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:12:23.089062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:12:23.089062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:12:23.089062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:12:23.089062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:12:23.089062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:12:23.089062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:12:23.089062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:12:23.089062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:12:23.089062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:12:23.090335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:12:23.090335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:12:23.090335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:12:23.090335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:12:23.090335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:12:23.090335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:12:23.090335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:12:23.090335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:12:23.090335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:12:23.090335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:12:23.090335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:12:23.090335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:12:23.090335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:12:23.090335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:12:23.090335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:12:23.090335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:12:23.091373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:12:23.091373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:12:23.091373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:12:23.091373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:12:23.091373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:12:23.091373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:12:23.091373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:12:23.091373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:12:23.091373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:12:23.091373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:12:23.091373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:23.091373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:12:23.091373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 570/1149] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:12:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:12:23.096167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b508d60-d41c-4c5c-b5d8-b0a7bb2ba3e2 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:23.192065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b508d60-d41c-4c5c-b5d8-b0a7bb2ba3e2 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:23.224966 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a3245-df8d-4dc1-b6b7-cd547df87382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:23.225642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a3245-df8d-4dc1-b6b7-cd547df87382 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:23.225929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a3245-df8d-4dc1-b6b7-cd547df87382 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:23.226208 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a3245-df8d-4dc1-b6b7-cd547df87382 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:23.226958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a3245-df8d-4dc1-b6b7-cd547df87382 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:23.227268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a3245-df8d-4dc1-b6b7-cd547df87382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:23.228671 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a3245-df8d-4dc1-b6b7-cd547df87382 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:23.236607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 580/1150] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:12:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:23.242316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a7715-993c-44cd-b9a0-49029a964b19 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:23.265623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:23.342224 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a7715-993c-44cd-b9a0-49029a964b19 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:23.374423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dcc9d-51b5-4f4d-8572-cb8302a1b0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:23.375291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dcc9d-51b5-4f4d-8572-cb8302a1b0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:23.375567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dcc9d-51b5-4f4d-8572-cb8302a1b0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:23.375836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dcc9d-51b5-4f4d-8572-cb8302a1b0c4 None admin] PATH_INFO: `/v3/users/8f2a5b7a162e433593d177f42cc6a6e4` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:23.377037 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9dcc9d-51b5-4f4d-8572-cb8302a1b0c4 None admin] RBAC: Authorizing `identity:get_user(user_id=8f2a5b7a162e433593d177f42cc6a6e4)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:23.378977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9dcc9d-51b5-4f4d-8572-cb8302a1b0c4 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:23.380613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 571/1151] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:12:23 2022] GET /identity/v3/users/8f2a5b7a162e433593d177f42cc6a6e4 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:23.386701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a3245-df8d-4dc1-b6b7-cd547df87382 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:23.406113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:23.484910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a3245-df8d-4dc1-b6b7-cd547df87382 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:23.516806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ff6bc-d7d7-4a62-b40b-d9fb1968dde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:23.517645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ff6bc-d7d7-4a62-b40b-d9fb1968dde6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:23.517925 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ff6bc-d7d7-4a62-b40b-d9fb1968dde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:23.518199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ff6bc-d7d7-4a62-b40b-d9fb1968dde6 None admin] PATH_INFO: `/v3/projects/5fc7b32810234a30a638de35a2abf536` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:23.519889 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ff6bc-d7d7-4a62-b40b-d9fb1968dde6 None admin] RBAC: Authorizing `identity:get_project(project_id=5fc7b32810234a30a638de35a2abf536)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:23.521329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ff6bc-d7d7-4a62-b40b-d9fb1968dde6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:23.522994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 581/1152] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:12:23 2022] GET /identity/v3/projects/5fc7b32810234a30a638de35a2abf536 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:23.528931 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dcc9d-51b5-4f4d-8572-cb8302a1b0c4 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:23.615591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:23.627852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dcc9d-51b5-4f4d-8572-cb8302a1b0c4 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:23.660629 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a751bb5-cdde-46ff-b1f9-3ed658669d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:23.661513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a751bb5-cdde-46ff-b1f9-3ed658669d03 None admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:23.661796 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a751bb5-cdde-46ff-b1f9-3ed658669d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:23.662070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a751bb5-cdde-46ff-b1f9-3ed658669d03 None admin] PATH_INFO: `/v3/projects/5fc7b32810234a30a638de35a2abf536/users/8f2a5b7a162e433593d177f42cc6a6e4/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:23.664013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a751bb5-cdde-46ff-b1f9-3ed658669d03 None admin] RBAC: Authorizing `identity:create_grant(project_id=5fc7b32810234a30a638de35a2abf536, user_id=8f2a5b7a162e433593d177f42cc6a6e4, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:23.665470 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a751bb5-cdde-46ff-b1f9-3ed658669d03 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:23.677857 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 572/1153] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:12:23 2022] PUT /identity/v3/projects/5fc7b32810234a30a638de35a2abf536/users/8f2a5b7a162e433593d177f42cc6a6e4/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:12:25.642091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79ebd0-5cf3-4666-8232-65c2f57821be None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:25.642620 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79ebd0-5cf3-4666-8232-65c2f57821be None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:25.642620 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79ebd0-5cf3-4666-8232-65c2f57821be None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:25.644567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 582/1154] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:25.649668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00724bd5-f07e-4ca1-bfe4-5080f8f09304 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:25.650060 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00724bd5-f07e-4ca1-bfe4-5080f8f09304 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:25.650224 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00724bd5-f07e-4ca1-bfe4-5080f8f09304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:25.666682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-00724bd5-f07e-4ca1-bfe4-5080f8f09304 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:25.667860 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-00724bd5-f07e-4ca1-bfe4-5080f8f09304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:25.732259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 573/1155] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:25.755795 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79ebd0-5cf3-4666-8232-65c2f57821be None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:25.839575 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:25.854938 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79ebd0-5cf3-4666-8232-65c2f57821be None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:25.887463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f680e-7216-44d4-8571-867579e7b6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:25.888238 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f680e-7216-44d4-8571-867579e7b6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:25.888550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f680e-7216-44d4-8571-867579e7b6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:25.888828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f680e-7216-44d4-8571-867579e7b6e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:25.899018 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f680e-7216-44d4-8571-867579e7b6e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:12:25.906537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-c60f680e-7216-44d4-8571-867579e7b6e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:25.906537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:12:25.906537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:12:25.906537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:12:25.906537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:12:25.906537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:12:25.906537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:12:25.906537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:12:25.906537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:12:25.906537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:12:25.906537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:12:25.906537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:12:25.906537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:12:25.906537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:12:25.906537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:12:25.906537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:12:25.907617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:12:25.907617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:12:25.907617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:12:25.907617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:12:25.907617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:12:25.907617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:12:25.907617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:12:25.907617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:12:25.907617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:12:25.907617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:12:25.907617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:12:25.907617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:12:25.907617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:12:25.907617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:12:25.907617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:12:25.907617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:12:25.908649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:12:25.908649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:12:25.908649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:12:25.908649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:12:25.908649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:12:25.908649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:12:25.908649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:12:25.908649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:12:25.908649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:12:25.908649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:12:25.908649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:25.908649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:12:25.908649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 583/1156] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:12:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:12:25.913731 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00724bd5-f07e-4ca1-bfe4-5080f8f09304 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:25.988182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:26.009914 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00724bd5-f07e-4ca1-bfe4-5080f8f09304 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:26.041666 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826adafd-2f2d-4ece-b6c2-8a9345c6caf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:26.042335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826adafd-2f2d-4ece-b6c2-8a9345c6caf8 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:26.042615 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826adafd-2f2d-4ece-b6c2-8a9345c6caf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:26.042912 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826adafd-2f2d-4ece-b6c2-8a9345c6caf8 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:26.043654 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826adafd-2f2d-4ece-b6c2-8a9345c6caf8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:26.043959 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826adafd-2f2d-4ece-b6c2-8a9345c6caf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:26.045342 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826adafd-2f2d-4ece-b6c2-8a9345c6caf8 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:26.053861 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 574/1157] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:12:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:26.059932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f680e-7216-44d4-8571-867579e7b6e8 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:26.155553 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f680e-7216-44d4-8571-867579e7b6e8 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:26.187781 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15379c4a-acb6-4edb-b187-b6c23b20a7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:26.188643 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15379c4a-acb6-4edb-b187-b6c23b20a7ef None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:26.188922 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15379c4a-acb6-4edb-b187-b6c23b20a7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:26.189192 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15379c4a-acb6-4edb-b187-b6c23b20a7ef None admin] PATH_INFO: `/v3/users/8f2a5b7a162e433593d177f42cc6a6e4` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:26.190844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15379c4a-acb6-4edb-b187-b6c23b20a7ef None admin] RBAC: Authorizing `identity:get_user(user_id=8f2a5b7a162e433593d177f42cc6a6e4)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:26.192261 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15379c4a-acb6-4edb-b187-b6c23b20a7ef None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:26.193874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 584/1158] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:12:26 2022] GET /identity/v3/users/8f2a5b7a162e433593d177f42cc6a6e4 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:26.199510 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826adafd-2f2d-4ece-b6c2-8a9345c6caf8 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:26.295394 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826adafd-2f2d-4ece-b6c2-8a9345c6caf8 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:26.327723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4fb35-b273-4cbf-9170-aab0e4723f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:26.328385 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4fb35-b273-4cbf-9170-aab0e4723f00 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:26.328669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4fb35-b273-4cbf-9170-aab0e4723f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:26.328943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4fb35-b273-4cbf-9170-aab0e4723f00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:26.339816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:26.394747 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4fb35-b273-4cbf-9170-aab0e4723f00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:26.396170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4fb35-b273-4cbf-9170-aab0e4723f00 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:26.491169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 575/1159] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:12:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:12:26.497192 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15379c4a-acb6-4edb-b187-b6c23b20a7ef None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:26.512934 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:26.594045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15379c4a-acb6-4edb-b187-b6c23b20a7ef None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:26.626529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764d330-abf6-4bdc-b882-0133ce36c581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:26.627368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764d330-abf6-4bdc-b882-0133ce36c581 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:26.627647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764d330-abf6-4bdc-b882-0133ce36c581 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:26.628481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764d330-abf6-4bdc-b882-0133ce36c581 None admin] PATH_INFO: `/v3/projects/5fc7b32810234a30a638de35a2abf536` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:26.629641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764d330-abf6-4bdc-b882-0133ce36c581 None admin] RBAC: Authorizing `identity:get_project(project_id=5fc7b32810234a30a638de35a2abf536)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:26.631059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764d330-abf6-4bdc-b882-0133ce36c581 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:26.632761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 585/1160] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:12:26 2022] GET /identity/v3/projects/5fc7b32810234a30a638de35a2abf536 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:26.638924 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4fb35-b273-4cbf-9170-aab0e4723f00 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:26.678810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:26.735544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4fb35-b273-4cbf-9170-aab0e4723f00 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:26.770233 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba985b8-72eb-414b-9298-6e3e783ba6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:26.770817 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba985b8-72eb-414b-9298-6e3e783ba6fd None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:26.771103 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba985b8-72eb-414b-9298-6e3e783ba6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:26.771381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba985b8-72eb-414b-9298-6e3e783ba6fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:26.772215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba985b8-72eb-414b-9298-6e3e783ba6fd None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=5fc7b32810234a30a638de35a2abf536, user.id=8f2a5b7a162e433593d177f42cc6a6e4) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:26.772560 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba985b8-72eb-414b-9298-6e3e783ba6fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:26.773982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba985b8-72eb-414b-9298-6e3e783ba6fd None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:26.779679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 576/1161] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:12:26 2022] GET /identity/v3/role_assignments?user.id=8f2a5b7a162e433593d177f42cc6a6e4&scope.project.id=5fc7b32810234a30a638de35a2abf536&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:28.797222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8d3d9-24f0-4459-ab0c-3c573a26b311 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:28.797750 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8d3d9-24f0-4459-ab0c-3c573a26b311 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:28.797750 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8d3d9-24f0-4459-ab0c-3c573a26b311 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:28.799066 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 586/1162] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:28.804264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43447bc8-531b-4520-be1d-e09834d89812 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:28.804721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43447bc8-531b-4520-be1d-e09834d89812 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:28.804828 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43447bc8-531b-4520-be1d-e09834d89812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:28.821199 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-43447bc8-531b-4520-be1d-e09834d89812 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:28.822479 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-43447bc8-531b-4520-be1d-e09834d89812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:28.887515 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 577/1163] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:28.909797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8d3d9-24f0-4459-ab0c-3c573a26b311 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:28.989760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:29.008398 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8d3d9-24f0-4459-ab0c-3c573a26b311 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:29.041582 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9d28d-d35e-4323-ab9a-d5d9d03a65ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:29.042215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9d28d-d35e-4323-ab9a-d5d9d03a65ee None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:29.042499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9d28d-d35e-4323-ab9a-d5d9d03a65ee None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:29.042775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9d28d-d35e-4323-ab9a-d5d9d03a65ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:29.106850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b9d28d-d35e-4323-ab9a-d5d9d03a65ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:29.108283 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b9d28d-d35e-4323-ab9a-d5d9d03a65ee None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:29.184195 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:29.202716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 587/1164] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:12:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:12:29.208687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43447bc8-531b-4520-be1d-e09834d89812 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:29.240774 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:29.304853 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43447bc8-531b-4520-be1d-e09834d89812 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:29.337355 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac55a4-f86f-4190-b103-af9cd083e0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:29.338153 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac55a4-f86f-4190-b103-af9cd083e0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:29.338430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac55a4-f86f-4190-b103-af9cd083e0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:29.338700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac55a4-f86f-4190-b103-af9cd083e0a4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:29.345431 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac55a4-f86f-4190-b103-af9cd083e0a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:29.347416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac55a4-f86f-4190-b103-af9cd083e0a4 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-92ac55a4-f86f-4190-b103-af9cd083e0a4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:12:29.361346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:12:29.363181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:12:29.363181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:12:29.363181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:12:29.363181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:12:29.363181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:12:29.363181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:12:29.363181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:12:29.363181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:12:29.363181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:12:29.363181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:12:29.363181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:12:29.363181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:12:29.363181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:12:29.363181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:12:29.363181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:12:29.363181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:12:29.364120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:12:29.364120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:12:29.364120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:12:29.364120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:12:29.364120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:12:29.364120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:12:29.364120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:12:29.364120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:12:29.364120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:12:29.364120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:12:29.364120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 20 19:12:29.364120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:12:29.364120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 19:12:29.364120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:12:29.364120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 578/1165] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jul 20 19:12:29 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:12:29.369056 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9d28d-d35e-4323-ab9a-d5d9d03a65ee None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:29.463632 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9d28d-d35e-4323-ab9a-d5d9d03a65ee None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:29.496758 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50df8cc-b063-4013-8ff6-9a6f0fcb7ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:29.497403 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50df8cc-b063-4013-8ff6-9a6f0fcb7ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:29.497686 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50df8cc-b063-4013-8ff6-9a6f0fcb7ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:29.497962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50df8cc-b063-4013-8ff6-9a6f0fcb7ed8 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:29.498682 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50df8cc-b063-4013-8ff6-9a6f0fcb7ed8 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:29.498992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50df8cc-b063-4013-8ff6-9a6f0fcb7ed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:29.500369 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50df8cc-b063-4013-8ff6-9a6f0fcb7ed8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:29.507331 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 588/1166] 10.222.0.43 () {56 vars in 1115 bytes} [Wed Jul 20 19:12:29 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:31.503483 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e4676-aa77-4ba0-a5b6-37c66bb8265d None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:31.504039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e4676-aa77-4ba0-a5b6-37c66bb8265d None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:31.504039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e4676-aa77-4ba0-a5b6-37c66bb8265d None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:31.505434 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 579/1167] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:31.510888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce63e2f-5279-4736-9f0b-013f445d129d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:31.511298 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce63e2f-5279-4736-9f0b-013f445d129d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:31.511298 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce63e2f-5279-4736-9f0b-013f445d129d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:31.528201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8ce63e2f-5279-4736-9f0b-013f445d129d None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:31.529397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8ce63e2f-5279-4736-9f0b-013f445d129d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:31.571017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:31.595579 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 589/1168] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:12:31.618025 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e4676-aa77-4ba0-a5b6-37c66bb8265d None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:31.709963 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:31.717373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e4676-aa77-4ba0-a5b6-37c66bb8265d None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:31.754621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d4850-87fa-4f78-b613-e8a9d49b2e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:31.755275 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d4850-87fa-4f78-b613-e8a9d49b2e36 None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:31.755562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d4850-87fa-4f78-b613-e8a9d49b2e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:31.755838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d4850-87fa-4f78-b613-e8a9d49b2e36 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:31.756625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d4850-87fa-4f78-b613-e8a9d49b2e36 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:31.757983 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d4850-87fa-4f78-b613-e8a9d49b2e36 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:31.774195 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 580/1169] 10.222.0.43 () {60 vars in 1138 bytes} [Wed Jul 20 19:12:31 2022] POST /identity/v3/domains => generated 267 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:12:33.813822 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3425a18-9547-4fa5-8d94-36a0b0937881 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:33.814359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3425a18-9547-4fa5-8d94-36a0b0937881 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:33.814359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3425a18-9547-4fa5-8d94-36a0b0937881 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:33.815741 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 590/1170] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:33.821052 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131401b-6226-4188-824b-eefbf85754df None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:33.821480 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131401b-6226-4188-824b-eefbf85754df None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:33.821612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131401b-6226-4188-824b-eefbf85754df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:33.838204 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-7131401b-6226-4188-824b-eefbf85754df None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:33.839406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-7131401b-6226-4188-824b-eefbf85754df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:33.904909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 581/1171] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:33.929126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3425a18-9547-4fa5-8d94-36a0b0937881 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:34.027779 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3425a18-9547-4fa5-8d94-36a0b0937881 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:34.060819 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1a3ab-1188-4e06-ab9f-b3387db431c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:34.061671 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1a3ab-1188-4e06-ab9f-b3387db431c8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:34.061949 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1a3ab-1188-4e06-ab9f-b3387db431c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:34.062219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1a3ab-1188-4e06-ab9f-b3387db431c8 None admin] PATH_INFO: `/v3/domains/2c7211ab550e4ad9a14d1607b8e3a370` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:34.072400 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1a3ab-1188-4e06-ab9f-b3387db431c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2c7211ab550e4ad9a14d1607b8e3a370)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:34.073770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1a3ab-1188-4e06-ab9f-b3387db431c8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:34.075489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 591/1172] 10.222.0.43 () {56 vars in 1183 bytes} [Wed Jul 20 19:12:33 2022] GET /identity/v3/domains/2c7211ab550e4ad9a14d1607b8e3a370 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:34.081927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131401b-6226-4188-824b-eefbf85754df None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:34.164653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:34.181648 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131401b-6226-4188-824b-eefbf85754df None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:34.215595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c7909-604d-4b69-a923-2f9cb9f694fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:34.216252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c7909-604d-4b69-a923-2f9cb9f694fa None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:34.216565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c7909-604d-4b69-a923-2f9cb9f694fa None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:34.216842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c7909-604d-4b69-a923-2f9cb9f694fa None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:34.217557 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c7909-604d-4b69-a923-2f9cb9f694fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:34.218980 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c7909-604d-4b69-a923-2f9cb9f694fa None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:34.243587 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 582/1173] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jul 20 19:12:34 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:12:36.318067 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fc66d-f237-40a1-9d04-df61dcff63e3 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:36.318659 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fc66d-f237-40a1-9d04-df61dcff63e3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:36.318659 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fc66d-f237-40a1-9d04-df61dcff63e3 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:36.320044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 592/1174] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:36.325133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f7a2a-63dc-4f8b-9fbb-678301cc74be None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:36.325630 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f7a2a-63dc-4f8b-9fbb-678301cc74be None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:36.325732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f7a2a-63dc-4f8b-9fbb-678301cc74be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:36.343415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-7a4f7a2a-63dc-4f8b-9fbb-678301cc74be None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:36.344629 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-7a4f7a2a-63dc-4f8b-9fbb-678301cc74be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:36.410819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 583/1175] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:36.435418 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fc66d-f237-40a1-9d04-df61dcff63e3 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:36.473117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:36.538394 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fc66d-f237-40a1-9d04-df61dcff63e3 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:36.571768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f36ae-791d-4352-ab7a-5a1839ee758a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:36.572748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f36ae-791d-4352-ab7a-5a1839ee758a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:36.573037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f36ae-791d-4352-ab7a-5a1839ee758a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:36.573319 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f36ae-791d-4352-ab7a-5a1839ee758a None admin] PATH_INFO: `/v3/domains/2c7211ab550e4ad9a14d1607b8e3a370` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:36.574505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f36ae-791d-4352-ab7a-5a1839ee758a None admin] RBAC: Authorizing `identity:get_domain(domain_id=2c7211ab550e4ad9a14d1607b8e3a370)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:36.575935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f36ae-791d-4352-ab7a-5a1839ee758a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:36.577590 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 593/1176] 10.222.0.43 () {56 vars in 1183 bytes} [Wed Jul 20 19:12:36 2022] GET /identity/v3/domains/2c7211ab550e4ad9a14d1607b8e3a370 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:36.584031 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f7a2a-63dc-4f8b-9fbb-678301cc74be None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:36.649255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:36.683878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f7a2a-63dc-4f8b-9fbb-678301cc74be None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:36.717850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe66ebc-a969-4baf-9b9a-0b12f5c34328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:36.718520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe66ebc-a969-4baf-9b9a-0b12f5c34328 None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:36.718803 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe66ebc-a969-4baf-9b9a-0b12f5c34328 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:36.719076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe66ebc-a969-4baf-9b9a-0b12f5c34328 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:36.719782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe66ebc-a969-4baf-9b9a-0b12f5c34328 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:36.721249 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe66ebc-a969-4baf-9b9a-0b12f5c34328 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:36.741319 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 584/1177] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:12:36 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:12:38.798117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac53e5-2307-48d7-8236-0e547dba8110 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:38.798779 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac53e5-2307-48d7-8236-0e547dba8110 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:38.798779 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac53e5-2307-48d7-8236-0e547dba8110 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:38.800041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 594/1178] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:12:38.805346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9c09e-1cdc-458c-a325-b99d63697944 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:38.805755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9c09e-1cdc-458c-a325-b99d63697944 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:38.805901 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9c09e-1cdc-458c-a325-b99d63697944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:38.822467 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-e4c9c09e-1cdc-458c-a325-b99d63697944 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:38.824222 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e4c9c09e-1cdc-458c-a325-b99d63697944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:38.890081 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 585/1179] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:12:38.914090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac53e5-2307-48d7-8236-0e547dba8110 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:39.021508 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac53e5-2307-48d7-8236-0e547dba8110 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:39.023754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:39.054062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67687c9-b303-4668-82f5-68ff32f3cfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:39.054977 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67687c9-b303-4668-82f5-68ff32f3cfd4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:39.055316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67687c9-b303-4668-82f5-68ff32f3cfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:39.055632 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67687c9-b303-4668-82f5-68ff32f3cfd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:39.066677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67687c9-b303-4668-82f5-68ff32f3cfd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:12:39.074275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-f67687c9-b303-4668-82f5-68ff32f3cfd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:39.074275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:12:39.074275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:12:39.074275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:12:39.074275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:12:39.074275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:12:39.074275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:12:39.074275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:12:39.074275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:12:39.074275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:12:39.074275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:12:39.074275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:12:39.074275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:12:39.074275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:12:39.074275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:12:39.074275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:12:39.075525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:12:39.075525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:12:39.075525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:12:39.075525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:12:39.075525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:12:39.075525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:12:39.075525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:12:39.075525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:12:39.075525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:12:39.075525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:12:39.075525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:12:39.075525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:12:39.075525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:12:39.075525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:12:39.075525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:12:39.075525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:12:39.076503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:12:39.076503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:12:39.076503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:12:39.076503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:12:39.076503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:12:39.076503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:12:39.076503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:12:39.076503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:12:39.076503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:12:39.076503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:12:39.076503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:39.076503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:12:39.076503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 595/1180] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:12:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:12:39.082543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9c09e-1cdc-458c-a325-b99d63697944 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:39.137760 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:39.184653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9c09e-1cdc-458c-a325-b99d63697944 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:39.218872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df88bd-b7dc-4853-903c-6d4f4de47728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:39.219541 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df88bd-b7dc-4853-903c-6d4f4de47728 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:39.219823 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df88bd-b7dc-4853-903c-6d4f4de47728 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:39.220101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df88bd-b7dc-4853-903c-6d4f4de47728 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:39.220884 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df88bd-b7dc-4853-903c-6d4f4de47728 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:39.221216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df88bd-b7dc-4853-903c-6d4f4de47728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:39.222580 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df88bd-b7dc-4853-903c-6d4f4de47728 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:39.230506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 586/1181] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:12:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:39.236392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67687c9-b303-4668-82f5-68ff32f3cfd4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:39.335178 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67687c9-b303-4668-82f5-68ff32f3cfd4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:39.370884 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50913d-75f3-4c29-b600-70f2d2b2dd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:39.371748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50913d-75f3-4c29-b600-70f2d2b2dd77 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:39.372026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50913d-75f3-4c29-b600-70f2d2b2dd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:39.372323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50913d-75f3-4c29-b600-70f2d2b2dd77 None admin] PATH_INFO: `/v3/users/514df7db280440c78566e415e28a238f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:39.386524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50913d-75f3-4c29-b600-70f2d2b2dd77 None admin] RBAC: Authorizing `identity:get_user(user_id=514df7db280440c78566e415e28a238f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:39.387963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50913d-75f3-4c29-b600-70f2d2b2dd77 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:39.389675 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 596/1182] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:12:39 2022] GET /identity/v3/users/514df7db280440c78566e415e28a238f => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:39.395390 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df88bd-b7dc-4853-903c-6d4f4de47728 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:39.494594 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df88bd-b7dc-4853-903c-6d4f4de47728 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:39.508264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:39.528582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab5bd7-2d13-4d11-aac8-a0f21ff5a372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:39.529229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab5bd7-2d13-4d11-aac8-a0f21ff5a372 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:39.529512 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab5bd7-2d13-4d11-aac8-a0f21ff5a372 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:39.529788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab5bd7-2d13-4d11-aac8-a0f21ff5a372 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:39.596123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab5bd7-2d13-4d11-aac8-a0f21ff5a372 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:39.597586 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab5bd7-2d13-4d11-aac8-a0f21ff5a372 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:39.695406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 587/1183] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:12:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:12:39.701236 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50913d-75f3-4c29-b600-70f2d2b2dd77 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:39.725347 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:39.799002 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50913d-75f3-4c29-b600-70f2d2b2dd77 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:39.833429 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b8d839-abc7-494b-9322-8af6f43dbc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:39.834271 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8d839-abc7-494b-9322-8af6f43dbc7b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:39.834550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8d839-abc7-494b-9322-8af6f43dbc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:39.834818 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8d839-abc7-494b-9322-8af6f43dbc7b None admin] PATH_INFO: `/v3/projects/eabe5572064543cfa5492f7fddc27cce` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:39.835991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b8d839-abc7-494b-9322-8af6f43dbc7b None admin] RBAC: Authorizing `identity:get_project(project_id=eabe5572064543cfa5492f7fddc27cce)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:39.837438 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b8d839-abc7-494b-9322-8af6f43dbc7b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:39.839139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 597/1184] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:12:39 2022] GET /identity/v3/projects/eabe5572064543cfa5492f7fddc27cce => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:39.846036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab5bd7-2d13-4d11-aac8-a0f21ff5a372 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:39.861495 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:39.943961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab5bd7-2d13-4d11-aac8-a0f21ff5a372 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:39.977680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54eaf56-20b8-45b3-a6da-2f021ae97bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:39.978323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54eaf56-20b8-45b3-a6da-2f021ae97bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:39.978608 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54eaf56-20b8-45b3-a6da-2f021ae97bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:39.978884 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54eaf56-20b8-45b3-a6da-2f021ae97bf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:39.979718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54eaf56-20b8-45b3-a6da-2f021ae97bf1 None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=eabe5572064543cfa5492f7fddc27cce, user.id=514df7db280440c78566e415e28a238f) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:39.980030 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54eaf56-20b8-45b3-a6da-2f021ae97bf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:39.981538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54eaf56-20b8-45b3-a6da-2f021ae97bf1 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:39.987439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 588/1185] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:12:39 2022] GET /identity/v3/role_assignments?user.id=514df7db280440c78566e415e28a238f&scope.project.id=eabe5572064543cfa5492f7fddc27cce&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:42.004921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d1c57-8993-4b29-85ac-ffcea2c26f40 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:42.005463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d1c57-8993-4b29-85ac-ffcea2c26f40 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:42.005463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d1c57-8993-4b29-85ac-ffcea2c26f40 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:42.006841 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 598/1186] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:42.012181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9396f-dc44-44fe-82d0-6d59430c87e0 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:42.012642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9396f-dc44-44fe-82d0-6d59430c87e0 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:42.012775 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9396f-dc44-44fe-82d0-6d59430c87e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:42.029619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-3ce9396f-dc44-44fe-82d0-6d59430c87e0 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:42.030815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-3ce9396f-dc44-44fe-82d0-6d59430c87e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:42.096858 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 589/1187] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:42.119344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d1c57-8993-4b29-85ac-ffcea2c26f40 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:42.208556 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:42.219200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d1c57-8993-4b29-85ac-ffcea2c26f40 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:42.251265 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf521f18-2b18-4b33-9205-3375ac572f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:42.252045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf521f18-2b18-4b33-9205-3375ac572f53 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:42.252358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf521f18-2b18-4b33-9205-3375ac572f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:42.253142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf521f18-2b18-4b33-9205-3375ac572f53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:42.263295 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf521f18-2b18-4b33-9205-3375ac572f53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:12:42.271509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-bf521f18-2b18-4b33-9205-3375ac572f53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:42.271509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:12:42.271509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:12:42.271509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:12:42.271509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:12:42.271509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:12:42.271509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:12:42.271509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:12:42.271509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:12:42.271509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:12:42.271509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:12:42.271509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:12:42.271509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:12:42.271509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:12:42.271509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:12:42.271509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:12:42.272693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:12:42.272693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:12:42.272693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:12:42.272693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:12:42.272693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:12:42.272693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:12:42.272693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:12:42.272693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:12:42.272693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:12:42.272693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:12:42.272693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:12:42.272693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:12:42.272693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:12:42.272693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:12:42.272693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:12:42.272693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:12:42.273612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:12:42.273612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:12:42.273612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:12:42.273612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:12:42.273612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:12:42.273612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:12:42.273612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:12:42.273612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:12:42.273612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:12:42.273612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:12:42.273612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:42.273612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:12:42.273612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 599/1188] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:12:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:12:42.278949 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9396f-dc44-44fe-82d0-6d59430c87e0 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:42.289092 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:42.379872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9396f-dc44-44fe-82d0-6d59430c87e0 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:42.412606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc861499-d678-4506-bf36-015bd2846b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:42.413271 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc861499-d678-4506-bf36-015bd2846b30 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:42.413551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc861499-d678-4506-bf36-015bd2846b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:42.413821 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc861499-d678-4506-bf36-015bd2846b30 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:42.414570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc861499-d678-4506-bf36-015bd2846b30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:42.414875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc861499-d678-4506-bf36-015bd2846b30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:42.416216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc861499-d678-4506-bf36-015bd2846b30 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:42.424037 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 590/1189] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:12:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:12:42.430646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf521f18-2b18-4b33-9205-3375ac572f53 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:42.530659 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf521f18-2b18-4b33-9205-3375ac572f53 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:42.564819 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70084a8b-47d4-4495-9b4f-f94b22088bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:42.566324 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70084a8b-47d4-4495-9b4f-f94b22088bdd None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:42.566611 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70084a8b-47d4-4495-9b4f-f94b22088bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:42.566886 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70084a8b-47d4-4495-9b4f-f94b22088bdd None admin] PATH_INFO: `/v3/users/514df7db280440c78566e415e28a238f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:42.568209 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70084a8b-47d4-4495-9b4f-f94b22088bdd None admin] RBAC: Authorizing `identity:get_user(user_id=514df7db280440c78566e415e28a238f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:42.569691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70084a8b-47d4-4495-9b4f-f94b22088bdd None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:42.571341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 600/1190] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:12:42 2022] GET /identity/v3/users/514df7db280440c78566e415e28a238f => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:42.577754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc861499-d678-4506-bf36-015bd2846b30 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:42.646404 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:42.677828 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc861499-d678-4506-bf36-015bd2846b30 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:42.710803 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95bbbe-94b0-495b-9c53-fac76870d278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:42.711657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95bbbe-94b0-495b-9c53-fac76870d278 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:42.711938 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95bbbe-94b0-495b-9c53-fac76870d278 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:42.712210 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95bbbe-94b0-495b-9c53-fac76870d278 None admin] PATH_INFO: `/v3/projects/eabe5572064543cfa5492f7fddc27cce` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:42.713406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95bbbe-94b0-495b-9c53-fac76870d278 None admin] RBAC: Authorizing `identity:get_project(project_id=eabe5572064543cfa5492f7fddc27cce)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:42.714917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95bbbe-94b0-495b-9c53-fac76870d278 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:42.716624 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 591/1191] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:12:42 2022] GET /identity/v3/projects/eabe5572064543cfa5492f7fddc27cce => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:42.724936 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70084a8b-47d4-4495-9b4f-f94b22088bdd None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:42.744753 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:42.822911 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70084a8b-47d4-4495-9b4f-f94b22088bdd None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:42.855410 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae825890-b41e-4bc8-b037-897020dec28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:42.857087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae825890-b41e-4bc8-b037-897020dec28a None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:42.857448 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae825890-b41e-4bc8-b037-897020dec28a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:42.857724 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae825890-b41e-4bc8-b037-897020dec28a None admin] PATH_INFO: `/v3/projects/eabe5572064543cfa5492f7fddc27cce/users/514df7db280440c78566e415e28a238f/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:42.859721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae825890-b41e-4bc8-b037-897020dec28a None admin] RBAC: Authorizing `identity:create_grant(project_id=eabe5572064543cfa5492f7fddc27cce, user_id=514df7db280440c78566e415e28a238f, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:42.861166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae825890-b41e-4bc8-b037-897020dec28a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:42.875344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 601/1192] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:12:42 2022] PUT /identity/v3/projects/eabe5572064543cfa5492f7fddc27cce/users/514df7db280440c78566e415e28a238f/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:12:44.856038 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac756a-a37e-4d48-96e5-eeca03b985ef None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:44.856625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac756a-a37e-4d48-96e5-eeca03b985ef None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:44.856625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac756a-a37e-4d48-96e5-eeca03b985ef None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:44.858000 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 592/1193] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:44.863150 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393b1f0-1f9f-4daf-bd2e-6b0829a70e6f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:44.863588 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393b1f0-1f9f-4daf-bd2e-6b0829a70e6f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:44.863681 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393b1f0-1f9f-4daf-bd2e-6b0829a70e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:44.880628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4393b1f0-1f9f-4daf-bd2e-6b0829a70e6f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:44.881808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4393b1f0-1f9f-4daf-bd2e-6b0829a70e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:44.946754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 602/1194] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:44.970371 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac756a-a37e-4d48-96e5-eeca03b985ef None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:45.073861 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac756a-a37e-4d48-96e5-eeca03b985ef None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:45.107129 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c3467-fa66-467c-a05b-0927eb09d5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:45.107839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c3467-fa66-467c-a05b-0927eb09d5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:45.108117 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c3467-fa66-467c-a05b-0927eb09d5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:45.108513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c3467-fa66-467c-a05b-0927eb09d5d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:45.115042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:45.119582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c3467-fa66-467c-a05b-0927eb09d5d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:12:45.128333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-337c3467-fa66-467c-a05b-0927eb09d5d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:45.128333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:12:45.128333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:12:45.128333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:12:45.128333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:12:45.128333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:12:45.128333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:12:45.128333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:12:45.128333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:12:45.128333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:12:45.128333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:12:45.128333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:12:45.128333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:12:45.128333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:12:45.128333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:12:45.128333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:12:45.129272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:12:45.129272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:12:45.129272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:12:45.129272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:12:45.129272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:12:45.129272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:12:45.129272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:12:45.129272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:12:45.129272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:12:45.129272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:12:45.129272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:12:45.129272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:12:45.129272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:12:45.129272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:12:45.129272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:12:45.129272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:12:45.130178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:12:45.130178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:12:45.130178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:12:45.130178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:12:45.130178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:12:45.130178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:12:45.130178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:12:45.130178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:12:45.130178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:12:45.130178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:12:45.130178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:45.130178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:12:45.130178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 593/1195] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:12:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:12:45.135631 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393b1f0-1f9f-4daf-bd2e-6b0829a70e6f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:45.146293 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:45.232157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393b1f0-1f9f-4daf-bd2e-6b0829a70e6f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:45.264561 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7788c5f-7efb-4bee-b22d-607381e8338f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:45.265216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7788c5f-7efb-4bee-b22d-607381e8338f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:45.265496 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7788c5f-7efb-4bee-b22d-607381e8338f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:45.265769 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7788c5f-7efb-4bee-b22d-607381e8338f None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:45.266514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7788c5f-7efb-4bee-b22d-607381e8338f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:45.266821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7788c5f-7efb-4bee-b22d-607381e8338f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:45.268808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7788c5f-7efb-4bee-b22d-607381e8338f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:45.276651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 603/1196] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:12:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:45.282447 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c3467-fa66-467c-a05b-0927eb09d5d1 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:45.383848 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c3467-fa66-467c-a05b-0927eb09d5d1 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:45.416182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ecc55-0b15-4c4a-a2e8-8138f4ce592f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:45.417360 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ecc55-0b15-4c4a-a2e8-8138f4ce592f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:45.417360 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ecc55-0b15-4c4a-a2e8-8138f4ce592f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:45.417624 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ecc55-0b15-4c4a-a2e8-8138f4ce592f None admin] PATH_INFO: `/v3/users/514df7db280440c78566e415e28a238f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:45.418834 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ecc55-0b15-4c4a-a2e8-8138f4ce592f None admin] RBAC: Authorizing `identity:get_user(user_id=514df7db280440c78566e415e28a238f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:45.420250 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ecc55-0b15-4c4a-a2e8-8138f4ce592f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:45.421903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 594/1197] 10.222.0.43 () {56 vars in 1177 bytes} [Wed Jul 20 19:12:45 2022] GET /identity/v3/users/514df7db280440c78566e415e28a238f => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:45.428823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7788c5f-7efb-4bee-b22d-607381e8338f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:45.497286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:45.529441 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7788c5f-7efb-4bee-b22d-607381e8338f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:45.562634 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1685f-003e-4661-b700-fcbf49f63fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:45.563271 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1685f-003e-4661-b700-fcbf49f63fde None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:45.563556 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1685f-003e-4661-b700-fcbf49f63fde None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:45.563831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1685f-003e-4661-b700-fcbf49f63fde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:45.629626 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1685f-003e-4661-b700-fcbf49f63fde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:45.631073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1685f-003e-4661-b700-fcbf49f63fde None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:45.696664 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:45.729683 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 604/1198] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:12:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:12:45.735623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ecc55-0b15-4c4a-a2e8-8138f4ce592f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:45.798864 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:45.835055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ecc55-0b15-4c4a-a2e8-8138f4ce592f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:45.868977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13abfed-403b-4bec-86b7-f608b60a4add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:45.869874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13abfed-403b-4bec-86b7-f608b60a4add None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:45.870483 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13abfed-403b-4bec-86b7-f608b60a4add None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:45.870483 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13abfed-403b-4bec-86b7-f608b60a4add None admin] PATH_INFO: `/v3/projects/eabe5572064543cfa5492f7fddc27cce` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:45.871635 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13abfed-403b-4bec-86b7-f608b60a4add None admin] RBAC: Authorizing `identity:get_project(project_id=eabe5572064543cfa5492f7fddc27cce)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:45.873162 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13abfed-403b-4bec-86b7-f608b60a4add None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:45.874845 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 595/1199] 10.222.0.43 () {56 vars in 1186 bytes} [Wed Jul 20 19:12:45 2022] GET /identity/v3/projects/eabe5572064543cfa5492f7fddc27cce => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:45.881072 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1685f-003e-4661-b700-fcbf49f63fde None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:45.976938 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1685f-003e-4661-b700-fcbf49f63fde None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:46.009088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980a937-eaf6-4743-ba2c-1893c552a168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:46.009730 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980a937-eaf6-4743-ba2c-1893c552a168 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:46.010008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980a937-eaf6-4743-ba2c-1893c552a168 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:46.010280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980a937-eaf6-4743-ba2c-1893c552a168 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:46.011105 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980a937-eaf6-4743-ba2c-1893c552a168 None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=eabe5572064543cfa5492f7fddc27cce, user.id=514df7db280440c78566e415e28a238f) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:46.011415 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980a937-eaf6-4743-ba2c-1893c552a168 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:46.012848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980a937-eaf6-4743-ba2c-1893c552a168 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:46.018947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 605/1200] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:12:45 2022] GET /identity/v3/role_assignments?user.id=514df7db280440c78566e415e28a238f&scope.project.id=eabe5572064543cfa5492f7fddc27cce&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:55.864581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=33 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:56.015837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=83 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:40.962462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01463ef5-f388-4b81-b10b-6b121f1b5b2d None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:40.962462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01463ef5-f388-4b81-b10b-6b121f1b5b2d None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:40.962462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01463ef5-f388-4b81-b10b-6b121f1b5b2d None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:40.962462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 596/1201] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:40.969732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52bf49-3fb8-4d5c-942e-494219340696 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:40.970211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52bf49-3fb8-4d5c-942e-494219340696 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:40.970296 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52bf49-3fb8-4d5c-942e-494219340696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:40.993025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8c52bf49-3fb8-4d5c-942e-494219340696 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:40.994601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8c52bf49-3fb8-4d5c-942e-494219340696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:41.088426 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 606/1202] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:13:41.115408 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01463ef5-f388-4b81-b10b-6b121f1b5b2d None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:41.249709 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01463ef5-f388-4b81-b10b-6b121f1b5b2d None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:41.296632 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81592d23-592e-4c44-9130-5b6425fc6265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:41.297534 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81592d23-592e-4c44-9130-5b6425fc6265 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:41.297797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81592d23-592e-4c44-9130-5b6425fc6265 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:41.298115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81592d23-592e-4c44-9130-5b6425fc6265 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:41.311540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81592d23-592e-4c44-9130-5b6425fc6265 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:41.313280 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81592d23-592e-4c44-9130-5b6425fc6265 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:41.334108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-81592d23-592e-4c44-9130-5b6425fc6265 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:41.334108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:41.334108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:41.334108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:41.334108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:41.334108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:41.334108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:41.334108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:41.334108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:41.334108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:41.334108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:41.334108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:41.334108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:41.334108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:41.334108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:41.335172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:41.336132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:41.336132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:41.336132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:41.336132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:13:41.340375 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 597/1203] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:41.341897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52bf49-3fb8-4d5c-942e-494219340696 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:41.484661 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52bf49-3fb8-4d5c-942e-494219340696 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:41.522001 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:41.529582 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83e02b-1557-4591-8f11-a0df9bf05258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:41.530286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83e02b-1557-4591-8f11-a0df9bf05258 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:41.530709 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83e02b-1557-4591-8f11-a0df9bf05258 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:41.530966 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83e02b-1557-4591-8f11-a0df9bf05258 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:41.531800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83e02b-1557-4591-8f11-a0df9bf05258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:41.532118 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83e02b-1557-4591-8f11-a0df9bf05258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:41.533646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83e02b-1557-4591-8f11-a0df9bf05258 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:41.551756 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 607/1204] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:41.556308 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81592d23-592e-4c44-9130-5b6425fc6265 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:41.634556 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:41.698560 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81592d23-592e-4c44-9130-5b6425fc6265 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:41.738932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509117f4-9644-4d8d-824d-df9f0e8f9ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:41.739685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509117f4-9644-4d8d-824d-df9f0e8f9ad4 None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:41.739893 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509117f4-9644-4d8d-824d-df9f0e8f9ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:41.740206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509117f4-9644-4d8d-824d-df9f0e8f9ad4 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:41.740965 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509117f4-9644-4d8d-824d-df9f0e8f9ad4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:41.742539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509117f4-9644-4d8d-824d-df9f0e8f9ad4 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:41.773213 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 598/1205] 10.222.0.43 () {60 vars in 1132 bytes} [Wed Jul 20 19:13:41 2022] POST /identity/v3/users => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:13:44.022482 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6a0af-274f-4b52-8b1b-55d6e43b722f None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:44.023028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6a0af-274f-4b52-8b1b-55d6e43b722f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:44.023114 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6a0af-274f-4b52-8b1b-55d6e43b722f None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:44.024808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 608/1206] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:13:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:44.031183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee34180-a49b-4f4d-b071-cc080e6ee627 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:44.031183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee34180-a49b-4f4d-b071-cc080e6ee627 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:44.032330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee34180-a49b-4f4d-b071-cc080e6ee627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:44.051306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-2ee34180-a49b-4f4d-b071-cc080e6ee627 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:44.052769 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-2ee34180-a49b-4f4d-b071-cc080e6ee627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:44.136585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 599/1207] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:13:44.161404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6a0af-274f-4b52-8b1b-55d6e43b722f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:44.279874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6a0af-274f-4b52-8b1b-55d6e43b722f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:44.318421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ae7e8-367d-44ec-9c63-2a5860d37963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:44.319221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ae7e8-367d-44ec-9c63-2a5860d37963 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:44.319542 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ae7e8-367d-44ec-9c63-2a5860d37963 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:44.319869 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ae7e8-367d-44ec-9c63-2a5860d37963 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:44.335346 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ae7e8-367d-44ec-9c63-2a5860d37963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:44.341831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-056ae7e8-367d-44ec-9c63-2a5860d37963 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:13:44.341831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:44.341831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:44.341831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:44.341831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:44.341831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:44.341831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:44.341831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:44.341831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:44.341831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:44.341831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:44.341831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:44.341831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:44.341831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:44.341831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:44.341831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:44.343008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:44.343008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:44.343008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:44.343008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:44.343008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:44.343008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:44.343008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:44.343008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:44.343008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:44.343008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:44.343008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:44.343008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:44.343008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:44.343008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:44.343008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:44.343008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:44.344065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:44.344065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:44.344065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:44.344065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:44.344065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:44.344065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:44.344065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:44.344065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:44.344065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:44.344065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:44.344065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:13:44.344065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:13:44.348608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 609/1208] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:13:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:44.350035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee34180-a49b-4f4d-b071-cc080e6ee627 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:44.394365 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:44.460622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee34180-a49b-4f4d-b071-cc080e6ee627 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:44.505927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038c6be-6e1c-41ab-b470-f9557327ec00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:44.506608 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038c6be-6e1c-41ab-b470-f9557327ec00 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:44.506889 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038c6be-6e1c-41ab-b470-f9557327ec00 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:44.507163 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038c6be-6e1c-41ab-b470-f9557327ec00 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:44.507924 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038c6be-6e1c-41ab-b470-f9557327ec00 None admin] RBAC: Adding query filter params (name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:44.508233 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038c6be-6e1c-41ab-b470-f9557327ec00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:44.509611 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038c6be-6e1c-41ab-b470-f9557327ec00 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:44.519305 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 600/1209] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:13:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:44.525773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ae7e8-367d-44ec-9c63-2a5860d37963 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:44.581425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:44.643733 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ae7e8-367d-44ec-9c63-2a5860d37963 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:44.680810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c4ab6-c5de-4bfb-989e-ab0298d337ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:44.681600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c4ab6-c5de-4bfb-989e-ab0298d337ce None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:44.681879 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c4ab6-c5de-4bfb-989e-ab0298d337ce None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:44.682410 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c4ab6-c5de-4bfb-989e-ab0298d337ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:44.693039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c4ab6-c5de-4bfb-989e-ab0298d337ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:44.694504 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c4ab6-c5de-4bfb-989e-ab0298d337ce None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:44.714198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-195c4ab6-c5de-4bfb-989e-ab0298d337ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:44.714198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:44.714198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:44.714198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:44.714198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:44.714198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:44.714198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:44.714198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:44.714198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:44.714198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:44.714198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:44.714198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:44.714198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:44.714198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:44.714198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:44.715214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:44.716194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:44.716194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:44.716194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:44.716194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:13:44.720313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 610/1210] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:44.721865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038c6be-6e1c-41ab-b470-f9557327ec00 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:44.844788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:44.848952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038c6be-6e1c-41ab-b470-f9557327ec00 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:44.889134 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a713187-a915-45b1-a15e-5daabb3ad2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:44.890665 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a713187-a915-45b1-a15e-5daabb3ad2fd None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:44.891084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a713187-a915-45b1-a15e-5daabb3ad2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:44.891469 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a713187-a915-45b1-a15e-5daabb3ad2fd None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:44.893196 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a713187-a915-45b1-a15e-5daabb3ad2fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:44.893664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a713187-a915-45b1-a15e-5daabb3ad2fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:44.895653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a713187-a915-45b1-a15e-5daabb3ad2fd None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:44.912061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 601/1211] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:44.918247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c4ab6-c5de-4bfb-989e-ab0298d337ce None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:45.040072 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c4ab6-c5de-4bfb-989e-ab0298d337ce None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:45.041709 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:45.080661 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba4428-8efd-4368-a62e-f4eb99e8eec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:45.083577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba4428-8efd-4368-a62e-f4eb99e8eec1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:45.083577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba4428-8efd-4368-a62e-f4eb99e8eec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:45.083577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba4428-8efd-4368-a62e-f4eb99e8eec1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:45.090695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba4428-8efd-4368-a62e-f4eb99e8eec1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:45.092138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba4428-8efd-4368-a62e-f4eb99e8eec1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:45.108059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-09ba4428-8efd-4368-a62e-f4eb99e8eec1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:45.108059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:45.108059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:45.108059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:45.108059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:45.108059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:45.108059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:45.108059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:45.108059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:45.108059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:45.108059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:45.108059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:45.108059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:13:45.108059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:13:45.108059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:13:45.108059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:45.109099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:45.110125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:45.110125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:45.110125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:13:45.110125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:13:45.110125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:13:45.110125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:13:45.110125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:13:45.110125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:13:45.110125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:45.110125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:13:45.110125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 611/1212] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:45.114134 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a713187-a915-45b1-a15e-5daabb3ad2fd None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:45.234762 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a713187-a915-45b1-a15e-5daabb3ad2fd None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:45.280528 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a774f-f072-486b-b673-2c5a461ca2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:45.281214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a774f-f072-486b-b673-2c5a461ca2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:45.281497 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a774f-f072-486b-b673-2c5a461ca2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:45.281774 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a774f-f072-486b-b673-2c5a461ca2d5 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:45.282619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a774f-f072-486b-b673-2c5a461ca2d5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:45.282955 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a774f-f072-486b-b673-2c5a461ca2d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:45.284399 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a774f-f072-486b-b673-2c5a461ca2d5 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:45.305436 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 602/1213] 10.222.0.43 () {56 vars in 1143 bytes} [Wed Jul 20 19:13:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:45.311161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba4428-8efd-4368-a62e-f4eb99e8eec1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:45.445124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba4428-8efd-4368-a62e-f4eb99e8eec1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:45.485276 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc7616-33e1-4690-95a0-72dd73d77c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:45.486440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc7616-33e1-4690-95a0-72dd73d77c57 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:45.486805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc7616-33e1-4690-95a0-72dd73d77c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:45.487094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc7616-33e1-4690-95a0-72dd73d77c57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:45.523924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:45.568603 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc7616-33e1-4690-95a0-72dd73d77c57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:45.570061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc7616-33e1-4690-95a0-72dd73d77c57 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:45.698243 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 612/1214] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 19:13:45.699530 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a774f-f072-486b-b673-2c5a461ca2d5 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:45.721605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:45.816401 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a774f-f072-486b-b673-2c5a461ca2d5 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:45.860645 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef10faa-a705-4c5f-9979-cf43e94ca707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:45.861997 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef10faa-a705-4c5f-9979-cf43e94ca707 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:45.862322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef10faa-a705-4c5f-9979-cf43e94ca707 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:45.862597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef10faa-a705-4c5f-9979-cf43e94ca707 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:45.873711 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef10faa-a705-4c5f-9979-cf43e94ca707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:45.875148 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef10faa-a705-4c5f-9979-cf43e94ca707 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:45.896976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-cef10faa-a705-4c5f-9979-cf43e94ca707 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:45.896976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:45.896976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:45.896976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:45.896976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:45.896976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:45.896976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:45.896976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:45.896976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:45.896976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:45.896976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:45.896976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:45.896976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:45.896976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:45.896976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:45.898042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:45.898944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:45.898944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:45.898944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:45.898944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:13:45.903077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 603/1215] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:45.904185 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc7616-33e1-4690-95a0-72dd73d77c57 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:45.976576 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:46.021511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc7616-33e1-4690-95a0-72dd73d77c57 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:46.063110 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517d1a2-6ed5-441e-b6d9-82bb9e8a6fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:46.063795 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517d1a2-6ed5-441e-b6d9-82bb9e8a6fec None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:46.064074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517d1a2-6ed5-441e-b6d9-82bb9e8a6fec None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:46.064386 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517d1a2-6ed5-441e-b6d9-82bb9e8a6fec None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:46.065123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517d1a2-6ed5-441e-b6d9-82bb9e8a6fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:46.065439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517d1a2-6ed5-441e-b6d9-82bb9e8a6fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:46.066864 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517d1a2-6ed5-441e-b6d9-82bb9e8a6fec None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:46.081885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 613/1216] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:46.087957 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef10faa-a705-4c5f-9979-cf43e94ca707 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:46.177468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:46.216580 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef10faa-a705-4c5f-9979-cf43e94ca707 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:46.258233 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc354b1-1d7e-403f-a12c-cd15de69419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:46.258966 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc354b1-1d7e-403f-a12c-cd15de69419b None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:46.259310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc354b1-1d7e-403f-a12c-cd15de69419b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:46.259690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc354b1-1d7e-403f-a12c-cd15de69419b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:46.279732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc354b1-1d7e-403f-a12c-cd15de69419b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:46.281292 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc354b1-1d7e-403f-a12c-cd15de69419b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:46.300098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-dbc354b1-1d7e-403f-a12c-cd15de69419b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:46.300098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:46.300098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:46.300098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:46.300098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:46.300098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:46.300098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:46.300098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:46.300098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:46.300098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:46.300098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:46.300098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:46.300098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:13:46.300098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:13:46.300098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:46.301125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:46.302013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:13:46.302013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:13:46.302013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:13:46.302013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:13:46.302013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:13:46.302013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:13:46.302013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:46.302013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:13:46.302013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 604/1217] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:13:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:46.309460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517d1a2-6ed5-441e-b6d9-82bb9e8a6fec None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:46.433232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517d1a2-6ed5-441e-b6d9-82bb9e8a6fec None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:46.462864 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:46.479463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883cc6d6-73a7-419b-aae8-97650c32d9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:46.480399 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cc6d6-73a7-419b-aae8-97650c32d9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:46.480801 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cc6d6-73a7-419b-aae8-97650c32d9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:46.481212 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cc6d6-73a7-419b-aae8-97650c32d9d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:46.482185 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883cc6d6-73a7-419b-aae8-97650c32d9d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:46.482660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883cc6d6-73a7-419b-aae8-97650c32d9d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:46.484309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883cc6d6-73a7-419b-aae8-97650c32d9d4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:46.503511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 614/1218] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:13:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:46.504784 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc354b1-1d7e-403f-a12c-cd15de69419b None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:46.617846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc354b1-1d7e-403f-a12c-cd15de69419b None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:46.650079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:46.657097 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b8d72-ad0e-4006-9613-199ab7341657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:46.657875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b8d72-ad0e-4006-9613-199ab7341657 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:46.658136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b8d72-ad0e-4006-9613-199ab7341657 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:46.658436 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b8d72-ad0e-4006-9613-199ab7341657 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:46.659366 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b8d72-ad0e-4006-9613-199ab7341657 None admin] RBAC: Adding query filter params (role.id=7fe62c5c771d41c28f790fade1df9357, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=ae581d3dda354e07894cb5205d1b2c5c) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:46.659679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b8d72-ad0e-4006-9613-199ab7341657 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:46.661225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b8d72-ad0e-4006-9613-199ab7341657 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:46.667487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 605/1219] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:46 2022] GET /identity/v3/role_assignments?user.id=ae581d3dda354e07894cb5205d1b2c5c&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=7fe62c5c771d41c28f790fade1df9357 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:48.878724 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88295308-d601-4116-9559-9d218e30710f None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:48.879341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88295308-d601-4116-9559-9d218e30710f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:48.879341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88295308-d601-4116-9559-9d218e30710f None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:48.880918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 615/1220] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:13:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:48.886205 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92eae2-9fa1-4c00-b9ca-eead645f6cf5 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:48.886693 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92eae2-9fa1-4c00-b9ca-eead645f6cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:48.886797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92eae2-9fa1-4c00-b9ca-eead645f6cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:48.905429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-1b92eae2-9fa1-4c00-b9ca-eead645f6cf5 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:48.906627 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-1b92eae2-9fa1-4c00-b9ca-eead645f6cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:48.980350 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 606/1221] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:13:48.999135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88295308-d601-4116-9559-9d218e30710f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:49.107416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88295308-d601-4116-9559-9d218e30710f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:49.144229 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e41c3a-e19c-419d-9466-61894c8550b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:49.144989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e41c3a-e19c-419d-9466-61894c8550b0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:49.145315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e41c3a-e19c-419d-9466-61894c8550b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:49.145562 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e41c3a-e19c-419d-9466-61894c8550b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:49.155742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e41c3a-e19c-419d-9466-61894c8550b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:49.163442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-c7e41c3a-e19c-419d-9466-61894c8550b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:13:49.163442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:49.163442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:49.163442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:49.163442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:49.163442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:49.163442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:49.163442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:49.163442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:49.163442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:49.163442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:49.163442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:49.163442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:49.163442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:49.163442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:49.163442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:49.164558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:49.164558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:49.164558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:49.164558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:49.164558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:49.164558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:49.164558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:49.164558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:49.164558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:49.164558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:49.164558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:49.164558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:49.164558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:49.164558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:49.164558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:49.164558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:49.165625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:49.165625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:49.165625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:49.165625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:49.165625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:49.165625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:49.165625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:49.165625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:49.165625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:49.165625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:49.165625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:13:49.165625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:13:49.170072 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 616/1222] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:13:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:49.171094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92eae2-9fa1-4c00-b9ca-eead645f6cf5 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:49.285395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92eae2-9fa1-4c00-b9ca-eead645f6cf5 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:49.313662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:49.325866 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6f668-9be5-4951-a052-8b6f6295e220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:49.326542 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6f668-9be5-4951-a052-8b6f6295e220 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:49.326824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6f668-9be5-4951-a052-8b6f6295e220 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:49.327102 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6f668-9be5-4951-a052-8b6f6295e220 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:49.327863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6f668-9be5-4951-a052-8b6f6295e220 None admin] RBAC: Adding query filter params (name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:49.328179 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6f668-9be5-4951-a052-8b6f6295e220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:49.329582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6f668-9be5-4951-a052-8b6f6295e220 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:49.339008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 607/1223] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:13:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:49.346895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e41c3a-e19c-419d-9466-61894c8550b0 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:49.397307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:49.464920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e41c3a-e19c-419d-9466-61894c8550b0 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:49.502717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53566c2d-cc70-464a-84d2-8109ee14e245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:49.503654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53566c2d-cc70-464a-84d2-8109ee14e245 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:49.503930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53566c2d-cc70-464a-84d2-8109ee14e245 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:49.504277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53566c2d-cc70-464a-84d2-8109ee14e245 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:49.515765 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53566c2d-cc70-464a-84d2-8109ee14e245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:49.517705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53566c2d-cc70-464a-84d2-8109ee14e245 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:49.538805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-53566c2d-cc70-464a-84d2-8109ee14e245 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:49.538805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:49.538805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:49.538805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:49.538805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:49.538805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:49.538805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:49.538805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:49.538805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:49.538805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:49.538805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:49.538805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:49.538805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:49.538805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:49.538805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:49.539924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:49.541453 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:49.541453 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:49.541453 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:49.541453 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:13:49.541453 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 617/1224] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:49.547570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6f668-9be5-4951-a052-8b6f6295e220 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:49.662770 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6f668-9be5-4951-a052-8b6f6295e220 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:49.699247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9c518-1362-4eb8-ba7b-29d8e9cdf637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:49.699915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9c518-1362-4eb8-ba7b-29d8e9cdf637 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:49.700201 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9c518-1362-4eb8-ba7b-29d8e9cdf637 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:49.700508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9c518-1362-4eb8-ba7b-29d8e9cdf637 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:49.701259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9c518-1362-4eb8-ba7b-29d8e9cdf637 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:49.701547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9c518-1362-4eb8-ba7b-29d8e9cdf637 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:49.702900 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9c518-1362-4eb8-ba7b-29d8e9cdf637 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:49.716045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 608/1225] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:49.723642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53566c2d-cc70-464a-84d2-8109ee14e245 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:49.838396 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:49.845483 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53566c2d-cc70-464a-84d2-8109ee14e245 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:49.883868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01420fb6-01a7-42ab-b455-76fd2ec6b010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:49.884708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01420fb6-01a7-42ab-b455-76fd2ec6b010 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:49.884988 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01420fb6-01a7-42ab-b455-76fd2ec6b010 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:49.885257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01420fb6-01a7-42ab-b455-76fd2ec6b010 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:49.894697 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01420fb6-01a7-42ab-b455-76fd2ec6b010 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:49.896230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01420fb6-01a7-42ab-b455-76fd2ec6b010 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:49.911220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-01420fb6-01a7-42ab-b455-76fd2ec6b010 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:49.911220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:49.911220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:49.911220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:49.911220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:49.911220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:49.911220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:49.911220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:49.911220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:49.911220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:49.911220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:49.911220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:49.911220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:13:49.911220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:13:49.911220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:13:49.911220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:49.912237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:49.913262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:49.913262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:49.913262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:13:49.913262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:13:49.913262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:13:49.913262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:13:49.913262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:13:49.913262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:13:49.913262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:49.913262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:13:49.913262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 618/1226] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:49.919905 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9c518-1362-4eb8-ba7b-29d8e9cdf637 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:49.983482 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:50.044915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9c518-1362-4eb8-ba7b-29d8e9cdf637 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:50.087240 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625f44a-eb6b-46d3-ba78-9aab580dd9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:50.087984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625f44a-eb6b-46d3-ba78-9aab580dd9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.088281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625f44a-eb6b-46d3-ba78-9aab580dd9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.088644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625f44a-eb6b-46d3-ba78-9aab580dd9d9 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.089729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625f44a-eb6b-46d3-ba78-9aab580dd9d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:50.090125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625f44a-eb6b-46d3-ba78-9aab580dd9d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:50.091869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625f44a-eb6b-46d3-ba78-9aab580dd9d9 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:50.113993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 609/1227] 10.222.0.43 () {56 vars in 1143 bytes} [Wed Jul 20 19:13:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:50.119791 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01420fb6-01a7-42ab-b455-76fd2ec6b010 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:50.232403 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01420fb6-01a7-42ab-b455-76fd2ec6b010 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:50.268591 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654adb6-ba89-410a-bd0d-ef6e61f1ead4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:50.269426 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654adb6-ba89-410a-bd0d-ef6e61f1ead4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.269749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654adb6-ba89-410a-bd0d-ef6e61f1ead4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.270028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654adb6-ba89-410a-bd0d-ef6e61f1ead4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.281058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654adb6-ba89-410a-bd0d-ef6e61f1ead4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:50.282502 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654adb6-ba89-410a-bd0d-ef6e61f1ead4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:50.302033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-4654adb6-ba89-410a-bd0d-ef6e61f1ead4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:50.302033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:50.302033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:50.302033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:50.302033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:50.302033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:50.302033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:50.302033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:50.302033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:50.302033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:50.302033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:50.302033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:50.302033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:50.302033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:50.302033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:50.303143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:50.304230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:50.304230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:50.304230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:50.304230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:13:50.304230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 619/1228] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:50.309920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625f44a-eb6b-46d3-ba78-9aab580dd9d9 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:50.415917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625f44a-eb6b-46d3-ba78-9aab580dd9d9 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:50.418306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:50.457716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9287fc8-6b9e-4ab1-a596-86791ba473eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:50.458364 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9287fc8-6b9e-4ab1-a596-86791ba473eb None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.458646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9287fc8-6b9e-4ab1-a596-86791ba473eb None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.458918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9287fc8-6b9e-4ab1-a596-86791ba473eb None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.459652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9287fc8-6b9e-4ab1-a596-86791ba473eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:50.459940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9287fc8-6b9e-4ab1-a596-86791ba473eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:50.461322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9287fc8-6b9e-4ab1-a596-86791ba473eb None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:50.474224 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 610/1229] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:50.480251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654adb6-ba89-410a-bd0d-ef6e61f1ead4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:50.490154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:50.591040 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654adb6-ba89-410a-bd0d-ef6e61f1ead4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:50.627362 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d59aac-3939-4e27-b4d7-85c2ab181666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:50.628157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d59aac-3939-4e27-b4d7-85c2ab181666 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.628480 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d59aac-3939-4e27-b4d7-85c2ab181666 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.628754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d59aac-3939-4e27-b4d7-85c2ab181666 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.635789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d59aac-3939-4e27-b4d7-85c2ab181666 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:50.637266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d59aac-3939-4e27-b4d7-85c2ab181666 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:50.655215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-e3d59aac-3939-4e27-b4d7-85c2ab181666 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:50.655215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:50.655215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:50.655215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:50.655215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:50.655215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:50.655215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:50.655215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:50.655215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:50.655215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:50.655215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:50.655215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:50.655215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:13:50.655215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:13:50.655215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:50.656193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:50.657132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:13:50.657132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:13:50.657132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:13:50.657132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:13:50.657132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:13:50.657132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:13:50.657132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:50.657132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:13:50.657132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 620/1230] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:13:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:50.664183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9287fc8-6b9e-4ab1-a596-86791ba473eb None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:50.771214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9287fc8-6b9e-4ab1-a596-86791ba473eb None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:50.812978 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467398e-b696-4160-8a58-04a63c99d69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:50.813636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467398e-b696-4160-8a58-04a63c99d69f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.813918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467398e-b696-4160-8a58-04a63c99d69f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.814217 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467398e-b696-4160-8a58-04a63c99d69f None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.815005 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467398e-b696-4160-8a58-04a63c99d69f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:50.815319 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467398e-b696-4160-8a58-04a63c99d69f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:50.816818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467398e-b696-4160-8a58-04a63c99d69f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:50.831947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 611/1231] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:13:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:50.838590 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d59aac-3939-4e27-b4d7-85c2ab181666 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:50.910845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:50.947832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d59aac-3939-4e27-b4d7-85c2ab181666 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:50.983379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5261c1-3261-4506-900f-0fb18e4d0daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:50.984263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5261c1-3261-4506-900f-0fb18e4d0daf None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.984571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5261c1-3261-4506-900f-0fb18e4d0daf None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.984844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5261c1-3261-4506-900f-0fb18e4d0daf None admin] PATH_INFO: `/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/ae581d3dda354e07894cb5205d1b2c5c/roles/7fe62c5c771d41c28f790fade1df9357` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:51.000479 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5261c1-3261-4506-900f-0fb18e4d0daf None admin] RBAC: Authorizing `identity:create_grant(project_id=5e2a50ebbb48417c911df499d98a0157, user_id=ae581d3dda354e07894cb5205d1b2c5c, role_id=7fe62c5c771d41c28f790fade1df9357)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:51.002177 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5261c1-3261-4506-900f-0fb18e4d0daf None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:51.016464 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 621/1232] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:13:50 2022] PUT /identity/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/ae581d3dda354e07894cb5205d1b2c5c/roles/7fe62c5c771d41c28f790fade1df9357 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:13:53.128685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982c5ce-a4e8-4464-819a-6f64852683c1 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:53.129233 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982c5ce-a4e8-4464-819a-6f64852683c1 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:53.129233 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982c5ce-a4e8-4464-819a-6f64852683c1 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:53.130710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 612/1233] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:13:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:53.136353 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45a20a-b1c7-4fef-a1f6-14a73e3932cf None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:53.136768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45a20a-b1c7-4fef-a1f6-14a73e3932cf None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:53.136920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45a20a-b1c7-4fef-a1f6-14a73e3932cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:53.154957 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8b45a20a-b1c7-4fef-a1f6-14a73e3932cf None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:53.156222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8b45a20a-b1c7-4fef-a1f6-14a73e3932cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:53.236154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 622/1234] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:13:53.261646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982c5ce-a4e8-4464-819a-6f64852683c1 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:53.290495 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:53.380800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982c5ce-a4e8-4464-819a-6f64852683c1 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:53.416496 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47c35c-86f4-4230-b111-672b87621bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:53.417294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47c35c-86f4-4230-b111-672b87621bdf None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:53.417578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47c35c-86f4-4230-b111-672b87621bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:53.417856 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47c35c-86f4-4230-b111-672b87621bdf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:53.429159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47c35c-86f4-4230-b111-672b87621bdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:53.437345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-ff47c35c-86f4-4230-b111-672b87621bdf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:13:53.437345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:53.437345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:53.437345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:53.437345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:53.437345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:53.437345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:53.437345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:53.437345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:53.437345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:53.437345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:53.437345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:53.437345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:53.437345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:53.437345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:53.437345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:53.438573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:53.438573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:53.438573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:53.438573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:53.438573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:53.438573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:53.438573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:53.438573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:53.438573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:53.438573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:53.438573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:53.438573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:53.438573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:53.438573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:53.438573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:53.438573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:53.440094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:53.440094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:53.440094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:53.440094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:53.440094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:53.440094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:53.440094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:53.440094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:53.440094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:53.440094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:53.440094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:13:53.440094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:13:53.440094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 613/1235] 10.222.0.43 () {56 vars in 1102 bytes} [Wed Jul 20 19:13:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:13:53.445589 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45a20a-b1c7-4fef-a1f6-14a73e3932cf None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:53.497362 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:53.558591 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45a20a-b1c7-4fef-a1f6-14a73e3932cf None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:53.596400 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b168010-0d6a-4696-a583-189551879f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:53.597134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b168010-0d6a-4696-a583-189551879f02 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:53.597421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b168010-0d6a-4696-a583-189551879f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:53.597693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b168010-0d6a-4696-a583-189551879f02 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:53.598458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b168010-0d6a-4696-a583-189551879f02 None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:53.598794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b168010-0d6a-4696-a583-189551879f02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:53.600279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b168010-0d6a-4696-a583-189551879f02 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:53.609275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 623/1236] 10.222.0.43 () {56 vars in 1103 bytes} [Wed Jul 20 19:13:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:53.616167 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47c35c-86f4-4230-b111-672b87621bdf None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:53.713759 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:53.731335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47c35c-86f4-4230-b111-672b87621bdf None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:53.772973 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1956aa0-30c9-44a6-aafa-2ad97eeb5cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:53.773939 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1956aa0-30c9-44a6-aafa-2ad97eeb5cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:53.774224 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1956aa0-30c9-44a6-aafa-2ad97eeb5cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:53.774497 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1956aa0-30c9-44a6-aafa-2ad97eeb5cc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:53.785997 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1956aa0-30c9-44a6-aafa-2ad97eeb5cc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:53.787527 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1956aa0-30c9-44a6-aafa-2ad97eeb5cc4 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:53.806076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-d1956aa0-30c9-44a6-aafa-2ad97eeb5cc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:53.806076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:53.806076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:53.806076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:53.806076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:53.806076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:53.806076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:53.806076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:53.806076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:53.806076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:53.806076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:53.806076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:53.806076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:53.806076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:53.806076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:53.807465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:53.808695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:53.808695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:53.808695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:53.808695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:13:53.808695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 614/1237] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:53.814871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b168010-0d6a-4696-a583-189551879f02 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:53.926689 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b168010-0d6a-4696-a583-189551879f02 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:53.936836 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:53.966083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee6377-d8fe-4d62-b785-1b8c53277cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:53.966871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee6377-d8fe-4d62-b785-1b8c53277cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:53.967206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee6377-d8fe-4d62-b785-1b8c53277cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:53.967442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee6377-d8fe-4d62-b785-1b8c53277cc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:53.968166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee6377-d8fe-4d62-b785-1b8c53277cc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:53.968505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee6377-d8fe-4d62-b785-1b8c53277cc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:53.969980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee6377-d8fe-4d62-b785-1b8c53277cc4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:53.983713 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 624/1238] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:53.990403 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1956aa0-30c9-44a6-aafa-2ad97eeb5cc4 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:54.109622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1956aa0-30c9-44a6-aafa-2ad97eeb5cc4 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:54.148791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17589843-6aed-4ce7-ad87-a2540d59239d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:54.149696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17589843-6aed-4ce7-ad87-a2540d59239d None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:54.149988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17589843-6aed-4ce7-ad87-a2540d59239d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:54.150283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17589843-6aed-4ce7-ad87-a2540d59239d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:54.153900 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:54.159618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17589843-6aed-4ce7-ad87-a2540d59239d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:54.161543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17589843-6aed-4ce7-ad87-a2540d59239d None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:54.176827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-17589843-6aed-4ce7-ad87-a2540d59239d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:54.176827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:54.176827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:54.176827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:54.176827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:54.176827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:54.176827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:54.176827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:54.176827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:54.176827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:54.176827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:54.176827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:54.176827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:13:54.176827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:13:54.176827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:13:54.176827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:54.178019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:54.179216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:54.179216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:54.179216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:13:54.179216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:13:54.179216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:13:54.179216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:13:54.179216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:13:54.179216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:13:54.179216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:54.179216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:13:54.179216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 615/1239] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:54.186304 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee6377-d8fe-4d62-b785-1b8c53277cc4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:54.294774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee6377-d8fe-4d62-b785-1b8c53277cc4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:54.334587 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78d356-bcc4-4519-9cf0-4efc3d04d343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:54.335421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78d356-bcc4-4519-9cf0-4efc3d04d343 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:54.335809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78d356-bcc4-4519-9cf0-4efc3d04d343 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:54.336253 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78d356-bcc4-4519-9cf0-4efc3d04d343 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:54.337280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f78d356-bcc4-4519-9cf0-4efc3d04d343 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:54.337639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f78d356-bcc4-4519-9cf0-4efc3d04d343 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:54.339104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f78d356-bcc4-4519-9cf0-4efc3d04d343 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:54.360324 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 625/1240] 10.222.0.43 () {56 vars in 1143 bytes} [Wed Jul 20 19:13:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:54.366934 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17589843-6aed-4ce7-ad87-a2540d59239d None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:54.479230 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17589843-6aed-4ce7-ad87-a2540d59239d None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:54.516955 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a66592-44ce-4e67-9506-b9e134eb03d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:54.517638 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a66592-44ce-4e67-9506-b9e134eb03d6 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:54.517923 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a66592-44ce-4e67-9506-b9e134eb03d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:54.518201 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a66592-44ce-4e67-9506-b9e134eb03d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:54.588223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:54.597288 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a66592-44ce-4e67-9506-b9e134eb03d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:54.598783 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a66592-44ce-4e67-9506-b9e134eb03d6 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:54.709943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 616/1241] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:13:54.716217 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78d356-bcc4-4519-9cf0-4efc3d04d343 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:54.745905 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:54.821623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78d356-bcc4-4519-9cf0-4efc3d04d343 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:54.858828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcebe4e-5035-4a04-9e80-04e21fa996fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:54.859551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcebe4e-5035-4a04-9e80-04e21fa996fc None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:54.859811 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcebe4e-5035-4a04-9e80-04e21fa996fc None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:54.860106 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcebe4e-5035-4a04-9e80-04e21fa996fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:54.871623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcebe4e-5035-4a04-9e80-04e21fa996fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:54.873122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcebe4e-5035-4a04-9e80-04e21fa996fc None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:54.893959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-3bcebe4e-5035-4a04-9e80-04e21fa996fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:54.893959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:54.893959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:54.893959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:54.893959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:54.893959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:54.893959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:54.893959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:54.893959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:54.893959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:54.893959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:54.893959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:54.893959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:54.893959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:54.893959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:54.894943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:54.895854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:54.895854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:54.895854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:54.895854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:13:54.895854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 626/1242] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:13:54.901932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a66592-44ce-4e67-9506-b9e134eb03d6 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:55.008336 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:55.015609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a66592-44ce-4e67-9506-b9e134eb03d6 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:55.052793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c637a-7980-4299-be17-624564b73e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:55.053484 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c637a-7980-4299-be17-624564b73e28 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:55.053767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c637a-7980-4299-be17-624564b73e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:55.054040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c637a-7980-4299-be17-624564b73e28 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:55.054757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c637a-7980-4299-be17-624564b73e28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:55.055062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c637a-7980-4299-be17-624564b73e28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:55.056582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c637a-7980-4299-be17-624564b73e28 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:55.077106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 617/1243] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:55.078336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcebe4e-5035-4a04-9e80-04e21fa996fc None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:55.176337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:55.201147 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcebe4e-5035-4a04-9e80-04e21fa996fc None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:55.240884 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7de32-e153-443a-a2ef-3d3ac7258dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:55.242124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7de32-e153-443a-a2ef-3d3ac7258dfb None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:55.242430 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7de32-e153-443a-a2ef-3d3ac7258dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:55.242758 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7de32-e153-443a-a2ef-3d3ac7258dfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:55.250071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7de32-e153-443a-a2ef-3d3ac7258dfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:55.252217 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7de32-e153-443a-a2ef-3d3ac7258dfb None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:55.267142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-93e7de32-e153-443a-a2ef-3d3ac7258dfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:55.267142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:55.267142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:55.267142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:55.267142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:55.267142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:55.267142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:55.267142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:55.267142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:55.267142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:55.267142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:55.267142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:55.267142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:13:55.267142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:13:55.267142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:55.268334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:55.269466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:13:55.269466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:13:55.269466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:13:55.269466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:13:55.269466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:13:55.269466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:13:55.269466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:55.269466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:13:55.269466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 627/1244] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:13:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:55.274539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c637a-7980-4299-be17-624564b73e28 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:55.382564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c637a-7980-4299-be17-624564b73e28 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:55.420323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ac77a-f8b3-454b-bd5b-482c055c4bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:55.420974 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ac77a-f8b3-454b-bd5b-482c055c4bbe None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:55.421259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ac77a-f8b3-454b-bd5b-482c055c4bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:55.421532 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ac77a-f8b3-454b-bd5b-482c055c4bbe None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:55.422311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ac77a-f8b3-454b-bd5b-482c055c4bbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:55.422619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ac77a-f8b3-454b-bd5b-482c055c4bbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:55.424782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ac77a-f8b3-454b-bd5b-482c055c4bbe None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:55.441036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 618/1245] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:13:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:55.447497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7de32-e153-443a-a2ef-3d3ac7258dfb None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:55.571247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7de32-e153-443a-a2ef-3d3ac7258dfb None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:55.610099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd10833-e965-4031-a1fc-91d9ea3fdc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:55.610748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd10833-e965-4031-a1fc-91d9ea3fdc45 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:55.611651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd10833-e965-4031-a1fc-91d9ea3fdc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:55.611927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd10833-e965-4031-a1fc-91d9ea3fdc45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:55.612799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd10833-e965-4031-a1fc-91d9ea3fdc45 None admin] RBAC: Adding query filter params (role.id=7fe62c5c771d41c28f790fade1df9357, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=ae581d3dda354e07894cb5205d1b2c5c) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:55.613108 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd10833-e965-4031-a1fc-91d9ea3fdc45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:55.614614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd10833-e965-4031-a1fc-91d9ea3fdc45 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:55.621245 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 628/1246] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:55 2022] GET /identity/v3/role_assignments?user.id=ae581d3dda354e07894cb5205d1b2c5c&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=7fe62c5c771d41c28f790fade1df9357 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:57.908702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cd76a-0083-4faa-af7e-82252cce3cb2 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:57.909249 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cd76a-0083-4faa-af7e-82252cce3cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:57.909249 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cd76a-0083-4faa-af7e-82252cce3cb2 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:57.910607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 619/1247] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:57.916074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877e9a1-9dec-4eac-9dc7-cc717473cf7b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:57.916542 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877e9a1-9dec-4eac-9dc7-cc717473cf7b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:57.916668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877e9a1-9dec-4eac-9dc7-cc717473cf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:57.921870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:57.936205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2877e9a1-9dec-4eac-9dc7-cc717473cf7b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:57.937458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2877e9a1-9dec-4eac-9dc7-cc717473cf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:58.010561 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 629/1248] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:13:58.036136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4cd76a-0083-4faa-af7e-82252cce3cb2 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:58.056781 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:58.156427 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4cd76a-0083-4faa-af7e-82252cce3cb2 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:58.200431 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74be639-7061-4826-81ec-501fbebd189d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:58.201197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74be639-7061-4826-81ec-501fbebd189d None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:58.201481 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74be639-7061-4826-81ec-501fbebd189d None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:58.201753 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74be639-7061-4826-81ec-501fbebd189d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:58.212494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74be639-7061-4826-81ec-501fbebd189d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:58.220412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-c74be639-7061-4826-81ec-501fbebd189d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:58.220412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:58.220412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:58.220412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:58.220412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:58.220412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:58.220412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:58.220412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:58.220412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:58.220412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:58.220412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:58.220412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:58.220412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:58.220412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:58.220412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:58.220412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:58.221642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:58.221642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:58.221642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:58.221642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:58.221642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:58.221642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:58.221642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:58.221642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:58.221642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:58.221642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:58.221642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:58.221642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:58.221642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:58.221642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:58.221642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:58.221642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:58.222636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:58.222636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:58.222636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:58.222636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:58.222636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:58.222636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:58.222636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:58.222636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:58.222636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:58.222636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:58.222636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:58.222636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:13:58.222636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 620/1249] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:13:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:13:58.228499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877e9a1-9dec-4eac-9dc7-cc717473cf7b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:58.338647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877e9a1-9dec-4eac-9dc7-cc717473cf7b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:58.370445 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:58.378236 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398eeaf0-bea8-4e86-baba-79daa9ff642f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:58.379283 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398eeaf0-bea8-4e86-baba-79daa9ff642f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:58.379631 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398eeaf0-bea8-4e86-baba-79daa9ff642f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:58.380036 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398eeaf0-bea8-4e86-baba-79daa9ff642f None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:58.381337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398eeaf0-bea8-4e86-baba-79daa9ff642f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:58.381883 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398eeaf0-bea8-4e86-baba-79daa9ff642f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:58.383985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398eeaf0-bea8-4e86-baba-79daa9ff642f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:58.396328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 630/1250] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:58.399420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74be639-7061-4826-81ec-501fbebd189d None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:58.495760 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:58.519599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74be639-7061-4826-81ec-501fbebd189d None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:58.554933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30baf56e-973a-4854-bf8f-4d19364a45f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:58.555749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30baf56e-973a-4854-bf8f-4d19364a45f7 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:58.556005 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30baf56e-973a-4854-bf8f-4d19364a45f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:58.556342 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30baf56e-973a-4854-bf8f-4d19364a45f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:58.567541 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30baf56e-973a-4854-bf8f-4d19364a45f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:58.569014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30baf56e-973a-4854-bf8f-4d19364a45f7 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:58.585826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-30baf56e-973a-4854-bf8f-4d19364a45f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:58.585826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:58.585826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:58.585826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:58.585826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:58.585826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:58.585826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:58.585826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:58.585826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:58.585826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:58.585826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:58.585826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:58.585826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:58.585826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:58.585826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:58.586757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:58.588942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:58.588942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:58.588942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:58.588942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:13:58.588942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 621/1251] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:58.593193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398eeaf0-bea8-4e86-baba-79daa9ff642f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:58.703202 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398eeaf0-bea8-4e86-baba-79daa9ff642f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:58.737553 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464724f-5dca-4a77-92b1-afd764a3fe0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:58.738216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464724f-5dca-4a77-92b1-afd764a3fe0a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:58.738496 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464724f-5dca-4a77-92b1-afd764a3fe0a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:58.738769 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464724f-5dca-4a77-92b1-afd764a3fe0a None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:58.739494 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464724f-5dca-4a77-92b1-afd764a3fe0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:58.739799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464724f-5dca-4a77-92b1-afd764a3fe0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:58.741194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464724f-5dca-4a77-92b1-afd764a3fe0a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:58.754140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 631/1252] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:58.759994 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30baf56e-973a-4854-bf8f-4d19364a45f7 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:58.865019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30baf56e-973a-4854-bf8f-4d19364a45f7 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:58.894666 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:58.904200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186b59b-38a4-42d2-902a-3246d5ae5139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:58.905225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186b59b-38a4-42d2-902a-3246d5ae5139 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:58.905621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186b59b-38a4-42d2-902a-3246d5ae5139 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:58.905971 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186b59b-38a4-42d2-902a-3246d5ae5139 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:58.914439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186b59b-38a4-42d2-902a-3246d5ae5139 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:58.915956 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186b59b-38a4-42d2-902a-3246d5ae5139 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:58.935755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-b186b59b-38a4-42d2-902a-3246d5ae5139 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:58.935755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:58.935755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:58.935755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:58.935755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:58.935755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:58.935755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:58.935755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:58.935755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:58.935755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:58.935755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:58.935755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:58.935755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:13:58.935755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:13:58.935755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:13:58.935755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:58.937322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:58.938993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:58.938993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:58.938993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:13:58.938993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:13:58.938993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:13:58.938993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:13:58.938993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:13:58.938993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:13:58.938993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:58.938993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:13:58.938993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 622/1253] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:58.944232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464724f-5dca-4a77-92b1-afd764a3fe0a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:59.010366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:59.059654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464724f-5dca-4a77-92b1-afd764a3fe0a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:59.116049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93538db-ba92-4ab8-b1e7-1fac5a7d40a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:59.116991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93538db-ba92-4ab8-b1e7-1fac5a7d40a3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:59.117584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93538db-ba92-4ab8-b1e7-1fac5a7d40a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:59.117983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93538db-ba92-4ab8-b1e7-1fac5a7d40a3 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:59.119465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93538db-ba92-4ab8-b1e7-1fac5a7d40a3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:59.120034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93538db-ba92-4ab8-b1e7-1fac5a7d40a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:59.122491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93538db-ba92-4ab8-b1e7-1fac5a7d40a3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:59.152046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 632/1254] 10.222.0.43 () {56 vars in 1143 bytes} [Wed Jul 20 19:13:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:59.160739 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186b59b-38a4-42d2-902a-3246d5ae5139 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:59.275125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186b59b-38a4-42d2-902a-3246d5ae5139 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:59.312912 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcdf307-9fbe-4909-ae6f-f16f1f515299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:59.313564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdf307-9fbe-4909-ae6f-f16f1f515299 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:59.313850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdf307-9fbe-4909-ae6f-f16f1f515299 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:59.314150 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdf307-9fbe-4909-ae6f-f16f1f515299 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:59.375316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:59.389739 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcdf307-9fbe-4909-ae6f-f16f1f515299 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:59.391192 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcdf307-9fbe-4909-ae6f-f16f1f515299 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:59.511100 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 623/1255] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:13:59.519258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93538db-ba92-4ab8-b1e7-1fac5a7d40a3 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:59.628874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93538db-ba92-4ab8-b1e7-1fac5a7d40a3 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:59.643423 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:59.665876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378fa8e-43e5-4504-ba2b-fbcf87cf2662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:59.666760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378fa8e-43e5-4504-ba2b-fbcf87cf2662 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:59.667046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378fa8e-43e5-4504-ba2b-fbcf87cf2662 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:59.667317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378fa8e-43e5-4504-ba2b-fbcf87cf2662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:59.678529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378fa8e-43e5-4504-ba2b-fbcf87cf2662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:59.679994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378fa8e-43e5-4504-ba2b-fbcf87cf2662 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:59.698645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-a378fa8e-43e5-4504-ba2b-fbcf87cf2662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:59.698645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:59.698645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:59.698645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:59.698645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:59.698645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:59.698645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:59.698645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:59.698645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:59.698645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:59.698645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:59.698645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:59.698645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:59.698645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:59.698645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:59.699775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:59.701268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:59.701268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:59.701268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:59.701268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:13:59.701268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 633/1256] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:13:59.706335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcdf307-9fbe-4909-ae6f-f16f1f515299 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:59.793152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:59.806439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcdf307-9fbe-4909-ae6f-f16f1f515299 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:59.843872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba50848-0805-459e-b80e-3a7d25b12744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:59.845087 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba50848-0805-459e-b80e-3a7d25b12744 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:59.845426 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba50848-0805-459e-b80e-3a7d25b12744 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:59.845761 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba50848-0805-459e-b80e-3a7d25b12744 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:59.846681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba50848-0805-459e-b80e-3a7d25b12744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:59.847104 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba50848-0805-459e-b80e-3a7d25b12744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:59.848721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba50848-0805-459e-b80e-3a7d25b12744 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:59.864104 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 624/1257] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:59.870511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378fa8e-43e5-4504-ba2b-fbcf87cf2662 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:59.986773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378fa8e-43e5-4504-ba2b-fbcf87cf2662 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:00.023004 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50147e3-e0cc-47bc-a59f-27eae7797b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:00.023791 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50147e3-e0cc-47bc-a59f-27eae7797b3e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:00.024073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50147e3-e0cc-47bc-a59f-27eae7797b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:00.024370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50147e3-e0cc-47bc-a59f-27eae7797b3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:00.032356 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50147e3-e0cc-47bc-a59f-27eae7797b3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:00.034360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50147e3-e0cc-47bc-a59f-27eae7797b3e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:00.049948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-c50147e3-e0cc-47bc-a59f-27eae7797b3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:00.049948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:00.049948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:00.049948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:00.049948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:00.049948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:00.049948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:00.049948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:00.049948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:00.049948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:00.049948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:00.049948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:00.049948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:00.049948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:00.049948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:00.051117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:00.052205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:00.052205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:00.052205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:00.052205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:00.052205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:00.052205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:00.052205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:00.052205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:14:00.052763 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 634/1258] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:13:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:00.058556 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba50848-0805-459e-b80e-3a7d25b12744 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:00.177102 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba50848-0805-459e-b80e-3a7d25b12744 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:00.213134 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1647e-40a3-460c-b61b-80014c7c76ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:00.213865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1647e-40a3-460c-b61b-80014c7c76ca None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:00.214147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1647e-40a3-460c-b61b-80014c7c76ca None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:00.214549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1647e-40a3-460c-b61b-80014c7c76ca None admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:00.215359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1647e-40a3-460c-b61b-80014c7c76ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:00.215666 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1647e-40a3-460c-b61b-80014c7c76ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:00.217132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1647e-40a3-460c-b61b-80014c7c76ca None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:00.230538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 625/1259] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:00.237255 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50147e3-e0cc-47bc-a59f-27eae7797b3e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:00.264881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:00.344399 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50147e3-e0cc-47bc-a59f-27eae7797b3e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:00.382502 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21e7af-6e36-4d5e-89c2-786d0d356069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:00.383166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21e7af-6e36-4d5e-89c2-786d0d356069 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:00.383447 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21e7af-6e36-4d5e-89c2-786d0d356069 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:00.383719 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21e7af-6e36-4d5e-89c2-786d0d356069 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:00.384599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21e7af-6e36-4d5e-89c2-786d0d356069 None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=ae581d3dda354e07894cb5205d1b2c5c) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:00.384907 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21e7af-6e36-4d5e-89c2-786d0d356069 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:00.386352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21e7af-6e36-4d5e-89c2-786d0d356069 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:00.403503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 635/1260] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:00 2022] GET /identity/v3/role_assignments?user.id=ae581d3dda354e07894cb5205d1b2c5c&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:02.454268 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40808a6b-039b-433a-94e6-246668972ddf None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:02.454793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40808a6b-039b-433a-94e6-246668972ddf None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:02.454901 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40808a6b-039b-433a-94e6-246668972ddf None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:02.456238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 626/1261] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:02.461379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d90ec-e3f9-4afd-ab2a-13681ba73614 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:02.461793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d90ec-e3f9-4afd-ab2a-13681ba73614 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:02.461932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d90ec-e3f9-4afd-ab2a-13681ba73614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:02.479017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d95d90ec-e3f9-4afd-ab2a-13681ba73614 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:02.480187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d95d90ec-e3f9-4afd-ab2a-13681ba73614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:02.545285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 636/1262] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:14:02.567635 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40808a6b-039b-433a-94e6-246668972ddf None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:02.578395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:02.665379 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40808a6b-039b-433a-94e6-246668972ddf None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:02.698528 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a073234-8246-41de-b980-d5004b7bd220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:02.699304 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a073234-8246-41de-b980-d5004b7bd220 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:02.699584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a073234-8246-41de-b980-d5004b7bd220 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:02.699856 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a073234-8246-41de-b980-d5004b7bd220 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:02.710107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a073234-8246-41de-b980-d5004b7bd220 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:02.717646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-7a073234-8246-41de-b980-d5004b7bd220 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:14:02.717646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:02.717646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:02.717646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:02.717646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:02.717646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:02.717646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:02.717646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:02.717646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:02.717646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:02.717646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:02.717646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:02.717646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:02.717646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:02.717646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:02.717646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:02.718720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:02.718720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:02.718720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:02.718720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:02.718720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:02.718720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:02.718720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:02.718720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:02.718720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:02.718720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:02.718720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:02.718720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:02.718720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:02.718720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:02.718720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:02.718720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:02.719904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:02.719904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:02.719904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:02.719904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:02.719904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:02.719904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:02.719904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:02.719904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:02.719904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:02.719904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:02.719904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:14:02.719904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:14:02.719904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 627/1263] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:14:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:02.725501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d90ec-e3f9-4afd-ab2a-13681ba73614 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:02.743306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:02.821394 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d90ec-e3f9-4afd-ab2a-13681ba73614 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:02.854993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7b4cc-2042-4484-a4af-e248193d6e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:02.855652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7b4cc-2042-4484-a4af-e248193d6e38 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:02.855930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7b4cc-2042-4484-a4af-e248193d6e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:02.856200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7b4cc-2042-4484-a4af-e248193d6e38 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:02.856987 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7b4cc-2042-4484-a4af-e248193d6e38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:02.857290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7b4cc-2042-4484-a4af-e248193d6e38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:02.858635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7b4cc-2042-4484-a4af-e248193d6e38 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:02.866618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 637/1264] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:02.872421 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a073234-8246-41de-b980-d5004b7bd220 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:02.942371 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:02.975208 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a073234-8246-41de-b980-d5004b7bd220 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:03.007032 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04055508-bf1d-45b7-80f3-ee472570277a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:03.007810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04055508-bf1d-45b7-80f3-ee472570277a None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:03.008090 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04055508-bf1d-45b7-80f3-ee472570277a None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:03.008388 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04055508-bf1d-45b7-80f3-ee472570277a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:03.018987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04055508-bf1d-45b7-80f3-ee472570277a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:03.020437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04055508-bf1d-45b7-80f3-ee472570277a None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:03.036765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-04055508-bf1d-45b7-80f3-ee472570277a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:03.036765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:03.036765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:03.036765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:03.036765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:03.036765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:03.036765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:03.036765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:03.036765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:03.036765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:03.036765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:03.036765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:03.036765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:03.036765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:03.036765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:03.037804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:03.038933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:03.038933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:03.038933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:03.038933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:14:03.038933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 628/1265] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:03.043943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7b4cc-2042-4484-a4af-e248193d6e38 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:03.134762 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:03.155751 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7b4cc-2042-4484-a4af-e248193d6e38 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:03.190167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50535b-2b0b-4384-a17b-4fbdf0300747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:03.190814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50535b-2b0b-4384-a17b-4fbdf0300747 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:03.191094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50535b-2b0b-4384-a17b-4fbdf0300747 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:03.191366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50535b-2b0b-4384-a17b-4fbdf0300747 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:03.192083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50535b-2b0b-4384-a17b-4fbdf0300747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:03.192465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50535b-2b0b-4384-a17b-4fbdf0300747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:03.193779 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50535b-2b0b-4384-a17b-4fbdf0300747 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:03.205667 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 638/1266] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:03.211581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04055508-bf1d-45b7-80f3-ee472570277a None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:03.311059 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04055508-bf1d-45b7-80f3-ee472570277a None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:03.330238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:03.343859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39004fc2-2396-4aaa-9c5f-a95602b9a78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:03.344690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39004fc2-2396-4aaa-9c5f-a95602b9a78f None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:03.344970 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39004fc2-2396-4aaa-9c5f-a95602b9a78f None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:03.345356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39004fc2-2396-4aaa-9c5f-a95602b9a78f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:03.353020 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39004fc2-2396-4aaa-9c5f-a95602b9a78f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:03.354446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39004fc2-2396-4aaa-9c5f-a95602b9a78f None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:03.369039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-39004fc2-2396-4aaa-9c5f-a95602b9a78f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:14:03.369039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:03.369039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:03.369039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:03.369039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:03.369039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:03.369039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:03.369039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:03.369039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:03.369039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:03.369039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:03.369039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:03.369039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:03.369039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:03.369039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:03.369039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:03.370202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:03.371180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:03.371180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:03.371180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:03.371180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:03.371180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:03.371180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:03.371180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:03.371180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:03.371180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:14:03.371180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:14:03.371180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 629/1267] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:03.376054 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50535b-2b0b-4384-a17b-4fbdf0300747 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:03.471907 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50535b-2b0b-4384-a17b-4fbdf0300747 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:03.504850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8233b4-c75d-4a00-a3bf-a18b79de0b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:03.505502 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8233b4-c75d-4a00-a3bf-a18b79de0b0e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:03.505783 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8233b4-c75d-4a00-a3bf-a18b79de0b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:03.506059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8233b4-c75d-4a00-a3bf-a18b79de0b0e None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:03.506886 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8233b4-c75d-4a00-a3bf-a18b79de0b0e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:03.507190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8233b4-c75d-4a00-a3bf-a18b79de0b0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:03.508715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8233b4-c75d-4a00-a3bf-a18b79de0b0e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:03.516970 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:03.525720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 639/1268] 10.222.0.43 () {56 vars in 1143 bytes} [Wed Jul 20 19:14:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:03.531332 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39004fc2-2396-4aaa-9c5f-a95602b9a78f None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:03.627717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39004fc2-2396-4aaa-9c5f-a95602b9a78f None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:03.659892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d1c00-ee68-4919-93cc-509f6694b359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:03.660698 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d1c00-ee68-4919-93cc-509f6694b359 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:03.660980 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d1c00-ee68-4919-93cc-509f6694b359 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:03.661252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d1c00-ee68-4919-93cc-509f6694b359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:03.671255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d1c00-ee68-4919-93cc-509f6694b359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:03.672689 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d1c00-ee68-4919-93cc-509f6694b359 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:03.689800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-c32d1c00-ee68-4919-93cc-509f6694b359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:03.689800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:03.689800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:03.689800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:03.689800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:03.689800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:03.689800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:03.689800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:03.689800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:03.689800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:03.689800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:03.689800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:03.689800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:03.689800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:03.689800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:03.690836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:03.691969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:03.691969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:03.691969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:03.691969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:14:03.691969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 630/1269] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:03.696641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8233b4-c75d-4a00-a3bf-a18b79de0b0e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:03.792707 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8233b4-c75d-4a00-a3bf-a18b79de0b0e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:03.825655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62476b9-b455-422a-8750-feac6e1c7cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:03.826304 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62476b9-b455-422a-8750-feac6e1c7cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:03.826587 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62476b9-b455-422a-8750-feac6e1c7cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:03.826861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62476b9-b455-422a-8750-feac6e1c7cd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:03.827579 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62476b9-b455-422a-8750-feac6e1c7cd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:03.827886 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62476b9-b455-422a-8750-feac6e1c7cd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:03.829273 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62476b9-b455-422a-8750-feac6e1c7cd5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:03.840113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 640/1270] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:03.846248 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d1c00-ee68-4919-93cc-509f6694b359 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:03.877226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:03.945592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d1c00-ee68-4919-93cc-509f6694b359 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:03.978468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3aa2fb-29a6-4107-9b5a-3deb082c65af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:03.979257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3aa2fb-29a6-4107-9b5a-3deb082c65af None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:03.979540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3aa2fb-29a6-4107-9b5a-3deb082c65af None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:03.979812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3aa2fb-29a6-4107-9b5a-3deb082c65af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:03.987075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3aa2fb-29a6-4107-9b5a-3deb082c65af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:03.988281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3aa2fb-29a6-4107-9b5a-3deb082c65af None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:04.002136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-bd3aa2fb-29a6-4107-9b5a-3deb082c65af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:04.002136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:04.002136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:04.002136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:04.002136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:04.002136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:04.002136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:04.002136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:04.002136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:04.002136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:04.002136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:04.002136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:04.002136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:04.002136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:04.002136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:04.003147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:04.004413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:04.004413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:04.004413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:04.004413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:04.004413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:04.004413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:04.004413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:04.004413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:14:04.004413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 631/1271] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:04.009179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62476b9-b455-422a-8750-feac6e1c7cd5 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:04.067923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:04.116047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62476b9-b455-422a-8750-feac6e1c7cd5 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:04.148337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef13563-da31-4173-9378-0c4a780b9831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:04.148988 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef13563-da31-4173-9378-0c4a780b9831 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:04.149273 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef13563-da31-4173-9378-0c4a780b9831 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:04.149547 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef13563-da31-4173-9378-0c4a780b9831 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:04.150334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef13563-da31-4173-9378-0c4a780b9831 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:04.150644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef13563-da31-4173-9378-0c4a780b9831 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:04.152057 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef13563-da31-4173-9378-0c4a780b9831 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:04.164242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 641/1272] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:04.170217 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3aa2fb-29a6-4107-9b5a-3deb082c65af None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:04.257225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:04.266051 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3aa2fb-29a6-4107-9b5a-3deb082c65af None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:04.299125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17011980-1947-4efc-bf6f-1aab3e6a7c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:04.299997 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17011980-1947-4efc-bf6f-1aab3e6a7c9b None admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:04.300278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17011980-1947-4efc-bf6f-1aab3e6a7c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:04.300578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17011980-1947-4efc-bf6f-1aab3e6a7c9b None admin] PATH_INFO: `/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/ae581d3dda354e07894cb5205d1b2c5c/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:04.302483 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17011980-1947-4efc-bf6f-1aab3e6a7c9b None admin] RBAC: Authorizing `identity:create_grant(project_id=5e2a50ebbb48417c911df499d98a0157, user_id=ae581d3dda354e07894cb5205d1b2c5c, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:04.303898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17011980-1947-4efc-bf6f-1aab3e6a7c9b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:04.316132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 632/1273] 10.222.0.43 () {58 vars in 1439 bytes} [Wed Jul 20 19:14:04 2022] PUT /identity/v3/projects/5e2a50ebbb48417c911df499d98a0157/users/ae581d3dda354e07894cb5205d1b2c5c/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:14:06.320607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7359add-f99f-4a42-8562-d70a034f6590 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:06.321140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7359add-f99f-4a42-8562-d70a034f6590 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:06.321140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7359add-f99f-4a42-8562-d70a034f6590 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:06.322520 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 642/1274] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:06.327669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca9444-6185-4b0b-9988-5b93978451e8 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:06.328083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca9444-6185-4b0b-9988-5b93978451e8 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:06.328226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca9444-6185-4b0b-9988-5b93978451e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:06.346965 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-27ca9444-6185-4b0b-9988-5b93978451e8 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:06.348138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-27ca9444-6185-4b0b-9988-5b93978451e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:06.413610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 633/1275] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:14:06.437712 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7359add-f99f-4a42-8562-d70a034f6590 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:06.535869 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7359add-f99f-4a42-8562-d70a034f6590 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:06.552919 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:06.570280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559dc81-1729-4099-924c-a6c4700119e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:06.571113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559dc81-1729-4099-924c-a6c4700119e6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:06.571343 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559dc81-1729-4099-924c-a6c4700119e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:06.571619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559dc81-1729-4099-924c-a6c4700119e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:06.581452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559dc81-1729-4099-924c-a6c4700119e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:06.589055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-0559dc81-1729-4099-924c-a6c4700119e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:14:06.589055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:06.589055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:06.589055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:06.589055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:06.589055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:06.589055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:06.589055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:06.589055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:06.589055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:06.589055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:06.589055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:06.589055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:06.589055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:06.589055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:06.589055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:06.590123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:06.590123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:06.590123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:06.590123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:06.590123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:06.590123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:06.590123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:06.590123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:06.590123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:06.590123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:06.590123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:06.590123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:06.590123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:06.590123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:06.590123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:06.590123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:06.595799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:06.595799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:06.595799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:06.595799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:06.595799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:06.595799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:06.595799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:06.595799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:06.595799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:06.595799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:06.595799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:14:06.595799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:14:06.595799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 643/1276] 10.222.0.43 () {56 vars in 1096 bytes} [Wed Jul 20 19:14:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:06.596530 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca9444-6185-4b0b-9988-5b93978451e8 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:06.672997 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:06.691752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca9444-6185-4b0b-9988-5b93978451e8 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:06.724171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b82f8-f525-4e32-a747-cb32ed76b405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:06.724848 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b82f8-f525-4e32-a747-cb32ed76b405 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:06.725128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b82f8-f525-4e32-a747-cb32ed76b405 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:06.725395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b82f8-f525-4e32-a747-cb32ed76b405 None admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:06.726131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b82f8-f525-4e32-a747-cb32ed76b405 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:06.726431 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b82f8-f525-4e32-a747-cb32ed76b405 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:06.727771 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b82f8-f525-4e32-a747-cb32ed76b405 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:06.736790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 634/1277] 10.222.0.43 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:06.744945 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559dc81-1729-4099-924c-a6c4700119e6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:06.842350 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559dc81-1729-4099-924c-a6c4700119e6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:06.874785 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73528b-9bdf-43e4-8cca-04f7637e999c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:06.875557 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73528b-9bdf-43e4-8cca-04f7637e999c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:06.875840 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73528b-9bdf-43e4-8cca-04f7637e999c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:06.876114 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73528b-9bdf-43e4-8cca-04f7637e999c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:06.886200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73528b-9bdf-43e4-8cca-04f7637e999c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:06.887649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73528b-9bdf-43e4-8cca-04f7637e999c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:06.903994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-bb73528b-9bdf-43e4-8cca-04f7637e999c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:06.903994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:06.903994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:06.903994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:06.903994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:06.903994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:06.903994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:06.903994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:06.903994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:06.903994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:06.903994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:06.903994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:06.903994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:06.903994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:06.903994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:06.905123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:06.906306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:06.906306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:06.906306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:06.906306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:14:06.906306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 644/1278] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:06.911606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b82f8-f525-4e32-a747-cb32ed76b405 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:07.006895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b82f8-f525-4e32-a747-cb32ed76b405 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:07.040568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03eae3-61bc-4351-8c30-2024dd6e5fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:07.041242 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03eae3-61bc-4351-8c30-2024dd6e5fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:07.041522 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03eae3-61bc-4351-8c30-2024dd6e5fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:07.041794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03eae3-61bc-4351-8c30-2024dd6e5fb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:07.042519 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03eae3-61bc-4351-8c30-2024dd6e5fb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:07.042824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03eae3-61bc-4351-8c30-2024dd6e5fb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:07.044181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03eae3-61bc-4351-8c30-2024dd6e5fb7 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:07.056400 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 635/1279] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:07.062243 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73528b-9bdf-43e4-8cca-04f7637e999c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:07.100217 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:07.182859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73528b-9bdf-43e4-8cca-04f7637e999c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:07.216747 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a574b-1c44-4441-ace9-c2b90a0d9904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:07.217552 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a574b-1c44-4441-ace9-c2b90a0d9904 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:07.217834 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a574b-1c44-4441-ace9-c2b90a0d9904 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:07.218111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a574b-1c44-4441-ace9-c2b90a0d9904 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:07.226331 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a574b-1c44-4441-ace9-c2b90a0d9904 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:07.227782 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a574b-1c44-4441-ace9-c2b90a0d9904 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:07.241761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-538a574b-1c44-4441-ace9-c2b90a0d9904 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:14:07.241761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:07.241761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:07.241761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:07.241761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:07.241761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:07.241761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:07.241761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:07.241761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:07.241761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:07.241761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:07.241761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:07.241761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:07.241761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:07.241761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:07.241761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:07.242917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:07.243903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:07.243903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:07.243903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:07.243903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:07.243903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:07.243903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:07.243903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:07.243903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:07.243903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:14:07.243903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:14:07.243903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 645/1280] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:07.248858 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03eae3-61bc-4351-8c30-2024dd6e5fb7 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:07.253006 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:07.347452 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03eae3-61bc-4351-8c30-2024dd6e5fb7 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:07.380590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784af17d-1cea-4e9b-ae5d-a1256a67ee01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:07.381254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784af17d-1cea-4e9b-ae5d-a1256a67ee01 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:07.381535 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784af17d-1cea-4e9b-ae5d-a1256a67ee01 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:07.381810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784af17d-1cea-4e9b-ae5d-a1256a67ee01 None admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:07.382645 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784af17d-1cea-4e9b-ae5d-a1256a67ee01 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:07.382955 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784af17d-1cea-4e9b-ae5d-a1256a67ee01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:07.384397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784af17d-1cea-4e9b-ae5d-a1256a67ee01 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:07.401523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 636/1281] 10.222.0.43 () {56 vars in 1143 bytes} [Wed Jul 20 19:14:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:07.407263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a574b-1c44-4441-ace9-c2b90a0d9904 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:07.494295 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:07.504367 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a574b-1c44-4441-ace9-c2b90a0d9904 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:07.536888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203b5c8-6a1e-4e72-9e14-193c7946b7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:07.537526 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203b5c8-6a1e-4e72-9e14-193c7946b7ae None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:07.537810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203b5c8-6a1e-4e72-9e14-193c7946b7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:07.538085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203b5c8-6a1e-4e72-9e14-193c7946b7ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:07.603614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203b5c8-6a1e-4e72-9e14-193c7946b7ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:07.605005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203b5c8-6a1e-4e72-9e14-193c7946b7ae None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:07.689883 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:07.700094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 646/1282] 10.222.0.43 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:14:07.706562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784af17d-1cea-4e9b-ae5d-a1256a67ee01 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:07.764969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:07.803473 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784af17d-1cea-4e9b-ae5d-a1256a67ee01 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:07.836441 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d96bc-a343-4f3f-8406-cc710691a4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:07.837221 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d96bc-a343-4f3f-8406-cc710691a4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:07.838484 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d96bc-a343-4f3f-8406-cc710691a4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:07.838484 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d96bc-a343-4f3f-8406-cc710691a4f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:07.847481 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d96bc-a343-4f3f-8406-cc710691a4f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:07.848951 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d96bc-a343-4f3f-8406-cc710691a4f4 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:07.866583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-4a8d96bc-a343-4f3f-8406-cc710691a4f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:07.866583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:07.866583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:07.866583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:07.866583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:07.866583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:07.866583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:07.866583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:07.866583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:07.866583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:07.866583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:07.866583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:07.866583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:07.866583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:07.866583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:07.867634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:07.868968 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:07.868968 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:07.868968 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:07.868968 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:14:07.868968 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 637/1283] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:07.873436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203b5c8-6a1e-4e72-9e14-193c7946b7ae None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:07.968410 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203b5c8-6a1e-4e72-9e14-193c7946b7ae None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:08.003085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6e621-6811-436a-a4e6-a31ad2b440c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:08.003735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6e621-6811-436a-a4e6-a31ad2b440c0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:08.004023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6e621-6811-436a-a4e6-a31ad2b440c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:08.004312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6e621-6811-436a-a4e6-a31ad2b440c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:08.005044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6e621-6811-436a-a4e6-a31ad2b440c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:08.005351 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6e621-6811-436a-a4e6-a31ad2b440c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:08.006714 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6e621-6811-436a-a4e6-a31ad2b440c0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:08.017818 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 647/1284] 10.222.0.43 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:08.023458 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d96bc-a343-4f3f-8406-cc710691a4f4 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:08.124131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d96bc-a343-4f3f-8406-cc710691a4f4 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:08.138088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:08.158053 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34663224-bf9b-4c61-8b7f-85bb3d89e1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:08.158847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34663224-bf9b-4c61-8b7f-85bb3d89e1fa None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:08.159126 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34663224-bf9b-4c61-8b7f-85bb3d89e1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:08.159397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34663224-bf9b-4c61-8b7f-85bb3d89e1fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:08.165777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34663224-bf9b-4c61-8b7f-85bb3d89e1fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:08.167209 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34663224-bf9b-4c61-8b7f-85bb3d89e1fa None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:08.180929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application [None req-34663224-bf9b-4c61-8b7f-85bb3d89e1fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:08.180929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:08.180929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:08.180929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:08.180929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:08.180929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:08.180929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:08.180929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:08.180929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:08.180929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:08.180929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:08.180929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:08.180929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:08.180929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:08.180929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:08.181880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:08.183449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:08.183449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:08.183449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:08.183449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:08.183449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:08.183449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:08.183449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:08.183449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: ERROR keystone.server.flask.application Jul 20 19:14:08.183449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 638/1285] 10.222.0.43 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:08.188182 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6e621-6811-436a-a4e6-a31ad2b440c0 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:08.236818 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:08.285012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6e621-6811-436a-a4e6-a31ad2b440c0 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:08.317663 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2741d-e854-457f-856e-e8ab56a36f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:08.318311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2741d-e854-457f-856e-e8ab56a36f6d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:08.318595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2741d-e854-457f-856e-e8ab56a36f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:08.318867 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2741d-e854-457f-856e-e8ab56a36f6d None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:08.319652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2741d-e854-457f-856e-e8ab56a36f6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:08.319960 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2741d-e854-457f-856e-e8ab56a36f6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:08.321411 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2741d-e854-457f-856e-e8ab56a36f6d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:08.333035 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 648/1286] 10.222.0.43 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:08.339001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34663224-bf9b-4c61-8b7f-85bb3d89e1fa None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:08.436359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34663224-bf9b-4c61-8b7f-85bb3d89e1fa None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:08.468045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f2d00-72a0-4986-b62a-cb49fd4ce6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:08.468716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f2d00-72a0-4986-b62a-cb49fd4ce6cb None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:08.469001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f2d00-72a0-4986-b62a-cb49fd4ce6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:08.469279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f2d00-72a0-4986-b62a-cb49fd4ce6cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:08.470640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f2d00-72a0-4986-b62a-cb49fd4ce6cb None admin] RBAC: Adding query filter params (role.id=8c52108bb1d34ca2a1d0fc643095dae8, scope.project.id=5e2a50ebbb48417c911df499d98a0157, user.id=ae581d3dda354e07894cb5205d1b2c5c) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:08.470954 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f2d00-72a0-4986-b62a-cb49fd4ce6cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:08.472390 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f2d00-72a0-4986-b62a-cb49fd4ce6cb None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:08.478044 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 639/1287] 10.222.0.43 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:08 2022] GET /identity/v3/role_assignments?user.id=ae581d3dda354e07894cb5205d1b2c5c&scope.project.id=5e2a50ebbb48417c911df499d98a0157&role.id=8c52108bb1d34ca2a1d0fc643095dae8 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:10.496382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacd30e-75ad-4426-8ea8-ef7b7948db10 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:10.496933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacd30e-75ad-4426-8ea8-ef7b7948db10 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:10.496933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacd30e-75ad-4426-8ea8-ef7b7948db10 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:10.498290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 649/1288] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:10.503512 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b002c9a-6b8e-4c04-a348-a2102640b77d None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:10.503929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b002c9a-6b8e-4c04-a348-a2102640b77d None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:10.504071 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b002c9a-6b8e-4c04-a348-a2102640b77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:10.520606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-8b002c9a-6b8e-4c04-a348-a2102640b77d None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:10.521778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-8b002c9a-6b8e-4c04-a348-a2102640b77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:10.550602 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:10.586568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 640/1289] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:14:10.612876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacd30e-75ad-4426-8ea8-ef7b7948db10 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:10.714138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacd30e-75ad-4426-8ea8-ef7b7948db10 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:10.724791 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:10.747449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0416a94-060c-4ad3-9885-0adb0416494a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:10.748229 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0416a94-060c-4ad3-9885-0adb0416494a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:10.748535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0416a94-060c-4ad3-9885-0adb0416494a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:10.748807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0416a94-060c-4ad3-9885-0adb0416494a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:10.749448 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0416a94-060c-4ad3-9885-0adb0416494a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:10.750793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0416a94-060c-4ad3-9885-0adb0416494a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-c0416a94-060c-4ad3-9885-0adb0416494a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:10.762281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:10.763519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:10.763519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:10.763519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:10.763519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:10.763519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:10.763519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:10.763519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:14:10.763519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:14:10.763519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:14:10.763519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:14:10.763519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:10.763519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:10.763519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:10.763519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:10.763519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:10.763519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:10.764623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:10.764623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:10.764623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:10.764623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:10.764623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:10.764623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:10.764623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:10.764623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:10.764623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:10.764623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:10.764623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:14:10.764623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:14:10.764623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:14:10.764623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:14:10.764623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 650/1290] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jul 20 19:14:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:10.770145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b002c9a-6b8e-4c04-a348-a2102640b77d None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:10.866356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b002c9a-6b8e-4c04-a348-a2102640b77d None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:10.898885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5bc409-517a-43fb-b6d3-8afe00e046d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:10.899540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bc409-517a-43fb-b6d3-8afe00e046d8 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:10.899823 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bc409-517a-43fb-b6d3-8afe00e046d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:10.900099 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bc409-517a-43fb-b6d3-8afe00e046d8 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:10.900902 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5bc409-517a-43fb-b6d3-8afe00e046d8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:10.901210 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5bc409-517a-43fb-b6d3-8afe00e046d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:10.902560 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5bc409-517a-43fb-b6d3-8afe00e046d8 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:10.907241 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 641/1291] 10.222.0.43 () {56 vars in 1116 bytes} [Wed Jul 20 19:14:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:10.913445 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0416a94-060c-4ad3-9885-0adb0416494a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:11.009625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0416a94-060c-4ad3-9885-0adb0416494a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:11.042746 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e10c36-569e-4425-a693-c656b6ba81de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:11.043391 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e10c36-569e-4425-a693-c656b6ba81de None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:11.043673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e10c36-569e-4425-a693-c656b6ba81de None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:11.043949 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e10c36-569e-4425-a693-c656b6ba81de None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:11.044676 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e10c36-569e-4425-a693-c656b6ba81de None admin] RBAC: Adding query filter params (type=placement) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:11.044985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e10c36-569e-4425-a693-c656b6ba81de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:11.046481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e10c36-569e-4425-a693-c656b6ba81de None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:11.050851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 651/1292] 10.222.0.43 () {56 vars in 1116 bytes} [Wed Jul 20 19:14:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:13.065110 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966051c-3850-4388-b032-cf6514f6571e None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:13.065734 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966051c-3850-4388-b032-cf6514f6571e None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:13.065734 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966051c-3850-4388-b032-cf6514f6571e None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:13.067013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 642/1293] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:14:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:13.072189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bc16a-e124-418b-bf1c-0a0057059501 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:13.072637 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bc16a-e124-418b-bf1c-0a0057059501 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:13.072784 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bc16a-e124-418b-bf1c-0a0057059501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:13.090012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-233bc16a-e124-418b-bf1c-0a0057059501 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:13.091205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-233bc16a-e124-418b-bf1c-0a0057059501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:13.125651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:13.164610 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 652/1294] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:14:13.187790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966051c-3850-4388-b032-cf6514f6571e None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:13.218234 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:13.286138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966051c-3850-4388-b032-cf6514f6571e None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:13.318945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a097fe0-b170-4080-ac8b-26b91cecb89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:13.319533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a097fe0-b170-4080-ac8b-26b91cecb89b None admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:13.319815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a097fe0-b170-4080-ac8b-26b91cecb89b None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:13.320086 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a097fe0-b170-4080-ac8b-26b91cecb89b None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:13.320854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a097fe0-b170-4080-ac8b-26b91cecb89b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:13.322226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a097fe0-b170-4080-ac8b-26b91cecb89b None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:13.335004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 643/1295] 10.222.0.43 () {60 vars in 1141 bytes} [Wed Jul 20 19:14:13 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:14:15.332469 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847eca5-8ab2-4bba-a7a0-11e95aa2adeb None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:15.332998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847eca5-8ab2-4bba-a7a0-11e95aa2adeb None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:15.332998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847eca5-8ab2-4bba-a7a0-11e95aa2adeb None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:15.334416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 653/1296] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:15.339606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491c589-382f-4925-9ddf-e260760d3298 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:15.340053 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491c589-382f-4925-9ddf-e260760d3298 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:15.340149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491c589-382f-4925-9ddf-e260760d3298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:15.356468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-e491c589-382f-4925-9ddf-e260760d3298 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:15.357641 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e491c589-382f-4925-9ddf-e260760d3298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:15.427988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 644/1297] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:14:15.451034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847eca5-8ab2-4bba-a7a0-11e95aa2adeb None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:15.548000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847eca5-8ab2-4bba-a7a0-11e95aa2adeb None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:15.581530 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a43a0a-76b2-4240-8b1c-d39f5f0d5957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:15.582320 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a43a0a-76b2-4240-8b1c-d39f5f0d5957 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:15.582655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a43a0a-76b2-4240-8b1c-d39f5f0d5957 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:15.582880 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a43a0a-76b2-4240-8b1c-d39f5f0d5957 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:15.583536 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a43a0a-76b2-4240-8b1c-d39f5f0d5957 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:15.584935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a43a0a-76b2-4240-8b1c-d39f5f0d5957 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-e7a43a0a-76b2-4240-8b1c-d39f5f0d5957 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:15.596742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:15.597874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:15.597874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:15.597874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:15.597874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:15.597874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:15.597874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:15.597874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:14:15.597874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:14:15.597874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:14:15.597874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:14:15.597874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:15.597874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:15.597874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:15.597874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:15.597874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:15.597874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:15.599037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:15.599037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:15.599037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:15.599037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:15.599037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:15.599037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:15.599037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:15.599037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:15.599037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:15.599037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:15.599037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:14:15.599037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:14:15.599037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:14:15.599037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:14:15.599037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 654/1298] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jul 20 19:14:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:15.603905 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491c589-382f-4925-9ddf-e260760d3298 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:15.622444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:15.702630 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491c589-382f-4925-9ddf-e260760d3298 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:15.734982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fc167-07f8-401d-85cf-eb3fab2c9e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:15.736181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fc167-07f8-401d-85cf-eb3fab2c9e39 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:15.736492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fc167-07f8-401d-85cf-eb3fab2c9e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:15.736765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fc167-07f8-401d-85cf-eb3fab2c9e39 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:15.737464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fc167-07f8-401d-85cf-eb3fab2c9e39 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:15.737768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fc167-07f8-401d-85cf-eb3fab2c9e39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:15.739102 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fc167-07f8-401d-85cf-eb3fab2c9e39 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:15.744131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 645/1299] 10.222.0.43 () {56 vars in 1116 bytes} [Wed Jul 20 19:14:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:15.750113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a43a0a-76b2-4240-8b1c-d39f5f0d5957 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:15.781048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:15.848728 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a43a0a-76b2-4240-8b1c-d39f5f0d5957 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:15.881204 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87925550-9dc6-4ba2-8555-12022a8247e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:15.881842 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87925550-9dc6-4ba2-8555-12022a8247e4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:15.882121 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87925550-9dc6-4ba2-8555-12022a8247e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:15.882390 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87925550-9dc6-4ba2-8555-12022a8247e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:15.883166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87925550-9dc6-4ba2-8555-12022a8247e4 None admin] RBAC: Adding query filter params (interface=public, service_id=04930d0689b74357bb7c33984a07d7a8, region_id=RegionOne) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:15.883471 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87925550-9dc6-4ba2-8555-12022a8247e4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:15.884918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87925550-9dc6-4ba2-8555-12022a8247e4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:15.889753 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 655/1300] 10.222.0.43 () {56 vars in 1251 bytes} [Wed Jul 20 19:14:15 2022] GET /identity/v3/endpoints?service_id=04930d0689b74357bb7c33984a07d7a8&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:15.895288 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fc167-07f8-401d-85cf-eb3fab2c9e39 None admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:15.973649 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:15.991365 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fc167-07f8-401d-85cf-eb3fab2c9e39 None admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:16.023749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64667d4-9a62-4614-8291-b688d27093e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:16.024429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64667d4-9a62-4614-8291-b688d27093e5 None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:16.024741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64667d4-9a62-4614-8291-b688d27093e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:16.025086 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64667d4-9a62-4614-8291-b688d27093e5 None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:16.025728 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64667d4-9a62-4614-8291-b688d27093e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:16.027063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64667d4-9a62-4614-8291-b688d27093e5 None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:16.034209 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 646/1301] 10.222.0.43 () {56 vars in 1087 bytes} [Wed Jul 20 19:14:15 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:14:18.046672 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a4995-c8a5-4c39-b7a5-4ae6ab76ebc1 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:18.047211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a4995-c8a5-4c39-b7a5-4ae6ab76ebc1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:18.047211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a4995-c8a5-4c39-b7a5-4ae6ab76ebc1 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:18.048672 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 656/1302] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:18.053966 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee969713-144b-41c3-ab91-ad0bd2337a04 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:18.054409 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee969713-144b-41c3-ab91-ad0bd2337a04 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:18.054504 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee969713-144b-41c3-ab91-ad0bd2337a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:18.075237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-ee969713-144b-41c3-ab91-ad0bd2337a04 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:18.076668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ee969713-144b-41c3-ab91-ad0bd2337a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:18.155784 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 647/1303] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:18 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:14:18.178374 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a4995-c8a5-4c39-b7a5-4ae6ab76ebc1 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:18.274874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:18.282107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a4995-c8a5-4c39-b7a5-4ae6ab76ebc1 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:18.315288 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b70487-4027-40cb-b9e5-e28a3a2d2039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:18.316083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b70487-4027-40cb-b9e5-e28a3a2d2039 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:18.316394 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b70487-4027-40cb-b9e5-e28a3a2d2039 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:18.316672 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b70487-4027-40cb-b9e5-e28a3a2d2039 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:18.317322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b70487-4027-40cb-b9e5-e28a3a2d2039 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:18.318677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b70487-4027-40cb-b9e5-e28a3a2d2039 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-81b70487-4027-40cb-b9e5-e28a3a2d2039 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:18.330339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:18.331514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:18.331514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:18.331514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:18.331514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:18.331514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:18.331514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:18.331514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:14:18.331514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:14:18.331514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:14:18.331514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:14:18.331514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:18.331514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:18.331514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:18.331514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:18.331514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:18.331514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:18.332802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:18.332802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:18.332802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:18.332802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:18.332802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:18.332802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:18.332802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:18.332802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:18.332802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:18.332802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:18.332802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:14:18.332802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:14:18.332802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:14:18.332802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jul 20 19:14:18.332802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 657/1304] 10.222.0.43 () {56 vars in 1117 bytes} [Wed Jul 20 19:14:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:18.338669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee969713-144b-41c3-ab91-ad0bd2337a04 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:18.412678 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:18.436487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee969713-144b-41c3-ab91-ad0bd2337a04 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:18.469423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c529f2-8346-4230-a66b-ab59b8f71aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:18.470088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c529f2-8346-4230-a66b-ab59b8f71aac None admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:18.470373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c529f2-8346-4230-a66b-ab59b8f71aac None admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:18.470649 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c529f2-8346-4230-a66b-ab59b8f71aac None admin] PATH_INFO: `/v3/services` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:18.471359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c529f2-8346-4230-a66b-ab59b8f71aac None admin] RBAC: Adding query filter params (name=placement) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:18.471669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c529f2-8346-4230-a66b-ab59b8f71aac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:18.473054 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c529f2-8346-4230-a66b-ab59b8f71aac None admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:18.478065 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 648/1305] 10.222.0.43 () {56 vars in 1116 bytes} [Wed Jul 20 19:14:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:18.483848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b70487-4027-40cb-b9e5-e28a3a2d2039 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:18.581810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b70487-4027-40cb-b9e5-e28a3a2d2039 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:18.613933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b85e4-e1a7-4e76-b8bb-c3278a07c0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:18.614571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b85e4-e1a7-4e76-b8bb-c3278a07c0ee None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:18.614855 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b85e4-e1a7-4e76-b8bb-c3278a07c0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:18.615133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b85e4-e1a7-4e76-b8bb-c3278a07c0ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:18.615865 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b85e4-e1a7-4e76-b8bb-c3278a07c0ee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:18.617253 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b85e4-e1a7-4e76-b8bb-c3278a07c0ee None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:18.635025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 658/1306] 10.222.0.43 () {60 vars in 1144 bytes} [Wed Jul 20 19:14:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:14:28.475197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=30 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:28.631342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:21.193246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777d794-9f2d-4121-b839-5fa0e61b9a6a None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:21.194283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777d794-9f2d-4121-b839-5fa0e61b9a6a None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:21.194797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777d794-9f2d-4121-b839-5fa0e61b9a6a None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:21.200268 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 649/1307] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:15:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:21.201749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd236089-5e02-4134-bfec-998f60166d2c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:21.202168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd236089-5e02-4134-bfec-998f60166d2c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:21.202378 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd236089-5e02-4134-bfec-998f60166d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:21.285433 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-dd236089-5e02-4134-bfec-998f60166d2c None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:21.287142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-dd236089-5e02-4134-bfec-998f60166d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:21.452710 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 659/1308] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:15:21.478237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ad8f0-148f-4319-a587-587d58e6a879 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:21.478528 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ad8f0-148f-4319-a587-587d58e6a879 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:21.478811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ad8f0-148f-4319-a587-587d58e6a879 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:21.480386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 650/1309] 10.222.0.43 () {54 vars in 965 bytes} [Wed Jul 20 19:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:21.489803 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15ac52-226d-4501-9b98-57abfc9aafba None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:21.490014 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15ac52-226d-4501-9b98-57abfc9aafba None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:21.490288 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15ac52-226d-4501-9b98-57abfc9aafba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:21.602546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5b15ac52-226d-4501-9b98-57abfc9aafba None None] MFA Rules not processed for user `74327a10eaab43e0af02050da1a24f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:21.604595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5b15ac52-226d-4501-9b98-57abfc9aafba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:21.640701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:21.731032 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 660/1310] 10.222.0.43 () {58 vars in 1062 bytes} [Wed Jul 20 19:15:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:21.742546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ad8f0-148f-4319-a587-587d58e6a879 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:21.744591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-a98ad8f0-148f-4319-a587-587d58e6a879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:21.865221 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ad8f0-148f-4319-a587-587d58e6a879 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:21.943389 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a50fc4-c550-4453-a1a4-61ba21842921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:21.944128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a50fc4-c550-4453-a1a4-61ba21842921 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:21.944508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a50fc4-c550-4453-a1a4-61ba21842921 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:21.944848 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a50fc4-c550-4453-a1a4-61ba21842921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:21.946717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-a2a50fc4-c550-4453-a1a4-61ba21842921 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:22.004401 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:22.065106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a50fc4-c550-4453-a1a4-61ba21842921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:22.066905 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a50fc4-c550-4453-a1a4-61ba21842921 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:22.090248 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 651/1311] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:24.436178 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366d4ac-093c-4aa3-b96f-7422f97ca833 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:24.436752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366d4ac-093c-4aa3-b96f-7422f97ca833 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:24.436752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366d4ac-093c-4aa3-b96f-7422f97ca833 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:24.438343 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 661/1312] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:24.443955 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b09b3-4f9a-4db0-b72e-7269026854ff None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:24.444385 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b09b3-4f9a-4db0-b72e-7269026854ff None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:24.444588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b09b3-4f9a-4db0-b72e-7269026854ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:24.525698 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-d42b09b3-4f9a-4db0-b72e-7269026854ff None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:24.526916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d42b09b3-4f9a-4db0-b72e-7269026854ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:24.629843 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 652/1313] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:24.668796 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366d4ac-093c-4aa3-b96f-7422f97ca833 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:24.670447 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3366d4ac-093c-4aa3-b96f-7422f97ca833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:24.791650 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366d4ac-093c-4aa3-b96f-7422f97ca833 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:24.868005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:24.870882 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96fa3e-7e29-4a07-8b11-deefb12d8505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:24.871572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96fa3e-7e29-4a07-8b11-deefb12d8505 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:24.872185 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96fa3e-7e29-4a07-8b11-deefb12d8505 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:24.872802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96fa3e-7e29-4a07-8b11-deefb12d8505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:24.875130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7d96fa3e-7e29-4a07-8b11-deefb12d8505 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:25.006563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96fa3e-7e29-4a07-8b11-deefb12d8505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:25.008009 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96fa3e-7e29-4a07-8b11-deefb12d8505 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:25.026759 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 662/1314] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:27.327692 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0260ce-624b-406d-ac7c-6323fa132519 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:27.328214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0260ce-624b-406d-ac7c-6323fa132519 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:27.328214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0260ce-624b-406d-ac7c-6323fa132519 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:27.329583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 653/1315] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:27.334693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bec6e-ae68-4681-b648-7c5bf75dc809 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:27.335123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bec6e-ae68-4681-b648-7c5bf75dc809 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:27.335235 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bec6e-ae68-4681-b648-7c5bf75dc809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:27.398829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3e3bec6e-ae68-4681-b648-7c5bf75dc809 None None] MFA Rules not processed for user `ff6ae2c3a54f48faabac6aeb4ab6d81f`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:27.400043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3e3bec6e-ae68-4681-b648-7c5bf75dc809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:27.452664 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:27.496792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 663/1316] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:27 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:15:27.536512 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0260ce-624b-406d-ac7c-6323fa132519 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:27.538042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-3d0260ce-624b-406d-ac7c-6323fa132519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:27.547762 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:27.641128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0260ce-624b-406d-ac7c-6323fa132519 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:27.712797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33aa1a-b824-4374-86f2-5e8bf291a5d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:27.713442 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33aa1a-b824-4374-86f2-5e8bf291a5d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:27.713724 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33aa1a-b824-4374-86f2-5e8bf291a5d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:27.713997 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33aa1a-b824-4374-86f2-5e8bf291a5d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:27.715645 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-1a33aa1a-b824-4374-86f2-5e8bf291a5d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:27.775178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:27.821971 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33aa1a-b824-4374-86f2-5e8bf291a5d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:27.823386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33aa1a-b824-4374-86f2-5e8bf291a5d2 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:27.840788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 654/1317] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:27 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:30.403786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a8fbc-478f-4bf8-93c9-9af3a51e9a20 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:30.404328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a8fbc-478f-4bf8-93c9-9af3a51e9a20 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:30.404427 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a8fbc-478f-4bf8-93c9-9af3a51e9a20 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:30.405782 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 664/1318] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:30.411628 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db2b3a-1324-47f9-8864-21f5ac782e8e None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:30.412039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db2b3a-1324-47f9-8864-21f5ac782e8e None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:30.412183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db2b3a-1324-47f9-8864-21f5ac782e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:30.476543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-09db2b3a-1324-47f9-8864-21f5ac782e8e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:30.477756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-09db2b3a-1324-47f9-8864-21f5ac782e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:30.570741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 655/1319] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:15:30.610545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a8fbc-478f-4bf8-93c9-9af3a51e9a20 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:30.612155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ad1a8fbc-478f-4bf8-93c9-9af3a51e9a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:30.713355 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a8fbc-478f-4bf8-93c9-9af3a51e9a20 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:30.783483 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f54b9d-be20-4390-9b79-343a0de06197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:30.784120 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f54b9d-be20-4390-9b79-343a0de06197 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:30.784433 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f54b9d-be20-4390-9b79-343a0de06197 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:30.784701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f54b9d-be20-4390-9b79-343a0de06197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:30.786332 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c6f54b9d-be20-4390-9b79-343a0de06197 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:30.793545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:30.891381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f54b9d-be20-4390-9b79-343a0de06197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:30.893411 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f54b9d-be20-4390-9b79-343a0de06197 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:30.910887 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 665/1320] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:30 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:33.585682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f5f49-4263-493d-8b72-6bbf920fb102 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:33.586211 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f5f49-4263-493d-8b72-6bbf920fb102 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:33.586299 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f5f49-4263-493d-8b72-6bbf920fb102 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:33.587618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 656/1321] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:33.592922 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8c0c1-8e44-48af-81d2-1b4ce37bb933 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:33.593347 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8c0c1-8e44-48af-81d2-1b4ce37bb933 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:33.593489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8c0c1-8e44-48af-81d2-1b4ce37bb933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:33.659519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-49c8c0c1-8e44-48af-81d2-1b4ce37bb933 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:33.660704 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-49c8c0c1-8e44-48af-81d2-1b4ce37bb933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:33.718665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:33.762978 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 666/1322] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:33.802723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f5f49-4263-493d-8b72-6bbf920fb102 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:33.804477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f40f5f49-4263-493d-8b72-6bbf920fb102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:33.824813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:33.912854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f5f49-4263-493d-8b72-6bbf920fb102 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:33.987848 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe1bda-7936-4138-9774-c1068922a2a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:33.988532 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe1bda-7936-4138-9774-c1068922a2a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:33.988811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe1bda-7936-4138-9774-c1068922a2a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:33.989083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe1bda-7936-4138-9774-c1068922a2a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:33.990760 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c3fe1bda-7936-4138-9774-c1068922a2a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:34.062356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:34.102381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe1bda-7936-4138-9774-c1068922a2a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:34.103820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe1bda-7936-4138-9774-c1068922a2a5 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:34.122830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 657/1323] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:36.241623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe38f38-328d-4f98-8f07-6460b2021104 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:36.242139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe38f38-328d-4f98-8f07-6460b2021104 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:36.242139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe38f38-328d-4f98-8f07-6460b2021104 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:36.243653 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 667/1324] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:36.249187 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1933f-c896-4170-addc-dbc5101de418 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:36.249644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1933f-c896-4170-addc-dbc5101de418 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:36.249784 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1933f-c896-4170-addc-dbc5101de418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:36.314012 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-42e1933f-c896-4170-addc-dbc5101de418 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:36.315857 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-42e1933f-c896-4170-addc-dbc5101de418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:36.418604 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 658/1325] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:36.455966 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe38f38-328d-4f98-8f07-6460b2021104 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:36.457828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5fe38f38-328d-4f98-8f07-6460b2021104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:36.562842 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe38f38-328d-4f98-8f07-6460b2021104 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:36.640522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc64b6-f366-4b7a-ab5f-10edb1f2366f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:36.641162 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc64b6-f366-4b7a-ab5f-10edb1f2366f service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:36.641443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc64b6-f366-4b7a-ab5f-10edb1f2366f service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:36.641715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc64b6-f366-4b7a-ab5f-10edb1f2366f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:36.643361 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3ccc64b6-f366-4b7a-ab5f-10edb1f2366f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:36.660543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:36.754369 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc64b6-f366-4b7a-ab5f-10edb1f2366f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:36.755833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc64b6-f366-4b7a-ab5f-10edb1f2366f service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:36.781777 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 668/1326] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:38.858631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9526fe6-bf68-4a85-98f4-1a6e5dfa2ed7 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:38.859192 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9526fe6-bf68-4a85-98f4-1a6e5dfa2ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:38.859192 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9526fe6-bf68-4a85-98f4-1a6e5dfa2ed7 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:38.860584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 659/1327] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:38.866133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b39b59-60ce-4475-915e-88979bd354e1 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:38.866715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b39b59-60ce-4475-915e-88979bd354e1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:38.866715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b39b59-60ce-4475-915e-88979bd354e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:38.936006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a8b39b59-60ce-4475-915e-88979bd354e1 None None] MFA Rules not processed for user `ff6ae2c3a54f48faabac6aeb4ab6d81f`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:38.937251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a8b39b59-60ce-4475-915e-88979bd354e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:39.006254 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:39.038639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 669/1328] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:38 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:39.077748 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9526fe6-bf68-4a85-98f4-1a6e5dfa2ed7 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:39.079262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f9526fe6-bf68-4a85-98f4-1a6e5dfa2ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:39.110005 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:39.186787 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9526fe6-bf68-4a85-98f4-1a6e5dfa2ed7 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:39.260421 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae139b30-66b7-4586-874d-272940beb3dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:39.261077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae139b30-66b7-4586-874d-272940beb3dc service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:39.261362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae139b30-66b7-4586-874d-272940beb3dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:39.261639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae139b30-66b7-4586-874d-272940beb3dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:39.263295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ae139b30-66b7-4586-874d-272940beb3dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:39.355060 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:39.394291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae139b30-66b7-4586-874d-272940beb3dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:39.395737 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae139b30-66b7-4586-874d-272940beb3dc service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:39.417062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 660/1329] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:39 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:41.702172 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346845ed-6230-4952-b917-bd60a37e82f2 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:41.702739 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346845ed-6230-4952-b917-bd60a37e82f2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:41.702739 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346845ed-6230-4952-b917-bd60a37e82f2 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:41.704095 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 670/1330] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:41.709513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02267e15-2df7-4356-948d-02cf0921720c None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:41.709912 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02267e15-2df7-4356-948d-02cf0921720c None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:41.710070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02267e15-2df7-4356-948d-02cf0921720c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:41.773871 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-02267e15-2df7-4356-948d-02cf0921720c None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:41.775071 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-02267e15-2df7-4356-948d-02cf0921720c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:41.869599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 661/1331] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:41.904773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346845ed-6230-4952-b917-bd60a37e82f2 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:41.906304 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-346845ed-6230-4952-b917-bd60a37e82f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:42.009076 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346845ed-6230-4952-b917-bd60a37e82f2 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:42.082807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e2525-14ff-4c55-9159-7e0698938f60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:42.083454 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e2525-14ff-4c55-9159-7e0698938f60 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:42.083739 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e2525-14ff-4c55-9159-7e0698938f60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:42.084015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e2525-14ff-4c55-9159-7e0698938f60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:42.085703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d84e2525-14ff-4c55-9159-7e0698938f60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:42.111655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:42.191906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e2525-14ff-4c55-9159-7e0698938f60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:42.193365 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e2525-14ff-4c55-9159-7e0698938f60 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:42.211391 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 671/1332] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:44.284503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc62a25-ec6a-431d-998d-a7c34a7f729c None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:44.285081 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc62a25-ec6a-431d-998d-a7c34a7f729c None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:44.285081 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc62a25-ec6a-431d-998d-a7c34a7f729c None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:44.286467 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 662/1333] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:44.291517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7727c4-dd98-43d6-995f-e363e1ba8582 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:44.291910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7727c4-dd98-43d6-995f-e363e1ba8582 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:44.292077 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7727c4-dd98-43d6-995f-e363e1ba8582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:44.355409 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0e7727c4-dd98-43d6-995f-e363e1ba8582 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:44.356635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0e7727c4-dd98-43d6-995f-e363e1ba8582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:44.429141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:44.451336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 672/1334] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:44.491828 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc62a25-ec6a-431d-998d-a7c34a7f729c None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:44.493514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0bc62a25-ec6a-431d-998d-a7c34a7f729c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:44.532883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:44.595732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc62a25-ec6a-431d-998d-a7c34a7f729c None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:44.666073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252f1e2-3b25-46f4-a778-3a26e4afa735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:44.666717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252f1e2-3b25-46f4-a778-3a26e4afa735 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:44.667003 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252f1e2-3b25-46f4-a778-3a26e4afa735 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:44.667278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252f1e2-3b25-46f4-a778-3a26e4afa735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:44.668964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e252f1e2-3b25-46f4-a778-3a26e4afa735 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:44.749227 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:44.777411 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252f1e2-3b25-46f4-a778-3a26e4afa735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:44.778850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252f1e2-3b25-46f4-a778-3a26e4afa735 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:44.796407 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 663/1335] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:47.151120 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd33a0c-8748-4ee8-ab20-974133e74769 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.151644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd33a0c-8748-4ee8-ab20-974133e74769 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.151738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd33a0c-8748-4ee8-ab20-974133e74769 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.153087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 673/1336] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:47.158261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d767251-4897-43b1-afe4-de3925301a2c None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.158686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d767251-4897-43b1-afe4-de3925301a2c None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.158823 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d767251-4897-43b1-afe4-de3925301a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.222754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4d767251-4897-43b1-afe4-de3925301a2c None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:47.223952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4d767251-4897-43b1-afe4-de3925301a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:47.319052 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 664/1337] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:15:47.370152 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3defd3d-ca31-4cfc-a306-cfb281ece654 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.370436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3defd3d-ca31-4cfc-a306-cfb281ece654 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.370711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3defd3d-ca31-4cfc-a306-cfb281ece654 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.372096 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 674/1338] 10.222.0.43 () {54 vars in 965 bytes} [Wed Jul 20 19:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:47.378597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae91207-83ac-4255-a4fc-efae38ce7098 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.378888 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae91207-83ac-4255-a4fc-efae38ce7098 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.379166 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae91207-83ac-4255-a4fc-efae38ce7098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.430757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:47.453316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-6ae91207-83ac-4255-a4fc-efae38ce7098 None None] MFA Rules not processed for user `74327a10eaab43e0af02050da1a24f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:47.454548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6ae91207-83ac-4255-a4fc-efae38ce7098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:47.528977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 665/1339] 10.222.0.43 () {58 vars in 1062 bytes} [Wed Jul 20 19:15:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:15:47.537872 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3defd3d-ca31-4cfc-a306-cfb281ece654 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:47.539387 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b3defd3d-ca31-4cfc-a306-cfb281ece654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:47.641232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3defd3d-ca31-4cfc-a306-cfb281ece654 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:47.712757 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c9a63-c81d-47be-a04d-c55a0bbfb9e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:47.713402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c9a63-c81d-47be-a04d-c55a0bbfb9e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.713682 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c9a63-c81d-47be-a04d-c55a0bbfb9e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.713954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c9a63-c81d-47be-a04d-c55a0bbfb9e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.715620 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ae8c9a63-c81d-47be-a04d-c55a0bbfb9e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:47.751528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:47.825932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c9a63-c81d-47be-a04d-c55a0bbfb9e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:47.827356 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c9a63-c81d-47be-a04d-c55a0bbfb9e2 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:47.844823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 675/1340] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:52.430000 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4bb2e-1a4a-4a2c-a370-35e49560b0e2 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:52.430501 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4bb2e-1a4a-4a2c-a370-35e49560b0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:52.430501 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4bb2e-1a4a-4a2c-a370-35e49560b0e2 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:52.431857 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 666/1341] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:52.437365 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e724d8a-dbb8-4186-8af3-17e16832c9a2 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:52.437839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e724d8a-dbb8-4186-8af3-17e16832c9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:52.437935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e724d8a-dbb8-4186-8af3-17e16832c9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:52.505604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8e724d8a-dbb8-4186-8af3-17e16832c9a2 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:52.506804 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8e724d8a-dbb8-4186-8af3-17e16832c9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:52.594673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:52.606453 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 676/1342] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:52.646326 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4bb2e-1a4a-4a2c-a370-35e49560b0e2 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:52.647900 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-2ec4bb2e-1a4a-4a2c-a370-35e49560b0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:52.752272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4bb2e-1a4a-4a2c-a370-35e49560b0e2 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:52.789571 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:52.826474 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75071b2a-43d4-44d1-95ad-7aa9058fdc43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:52.827125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75071b2a-43d4-44d1-95ad-7aa9058fdc43 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:52.827403 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75071b2a-43d4-44d1-95ad-7aa9058fdc43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:52.827674 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75071b2a-43d4-44d1-95ad-7aa9058fdc43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:52.829347 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-75071b2a-43d4-44d1-95ad-7aa9058fdc43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:52.944399 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75071b2a-43d4-44d1-95ad-7aa9058fdc43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:52.945855 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75071b2a-43d4-44d1-95ad-7aa9058fdc43 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:52.964480 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 667/1343] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:15:55.647465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45586c8-0e4d-4860-9594-acefeda65a9b None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:55.648039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45586c8-0e4d-4860-9594-acefeda65a9b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:55.648039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45586c8-0e4d-4860-9594-acefeda65a9b None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:55.649490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 677/1344] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:55.655226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a4ff9-3e32-48ae-9be7-3726edc46114 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:55.655761 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a4ff9-3e32-48ae-9be7-3726edc46114 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:55.655761 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a4ff9-3e32-48ae-9be7-3726edc46114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:55.726916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-0f8a4ff9-3e32-48ae-9be7-3726edc46114 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:55.727487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0f8a4ff9-3e32-48ae-9be7-3726edc46114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:55.746791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:55.831615 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 668/1345] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:55.869028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45586c8-0e4d-4860-9594-acefeda65a9b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:55.870579 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b45586c8-0e4d-4860-9594-acefeda65a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:55.981425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45586c8-0e4d-4860-9594-acefeda65a9b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:56.054400 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068bbd3-ac26-4c65-90f1-4b6f0bbba91f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:56.055053 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068bbd3-ac26-4c65-90f1-4b6f0bbba91f service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:56.055371 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068bbd3-ac26-4c65-90f1-4b6f0bbba91f service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:56.055679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068bbd3-ac26-4c65-90f1-4b6f0bbba91f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:56.057308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2068bbd3-ac26-4c65-90f1-4b6f0bbba91f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:56.105538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:56.173101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2068bbd3-ac26-4c65-90f1-4b6f0bbba91f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:56.174552 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2068bbd3-ac26-4c65-90f1-4b6f0bbba91f service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:56.193287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 678/1346] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:00.173145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a974d10-f8ca-4eac-beef-c9da4e157251 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:00.174167 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a974d10-f8ca-4eac-beef-c9da4e157251 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:00.174284 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a974d10-f8ca-4eac-beef-c9da4e157251 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:00.175767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 669/1347] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:16:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:00.181171 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b76f3-e9c0-4e89-ac77-07f33cea8044 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:00.181761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b76f3-e9c0-4e89-ac77-07f33cea8044 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:00.181761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b76f3-e9c0-4e89-ac77-07f33cea8044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:00.248724 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e20b76f3-e9c0-4e89-ac77-07f33cea8044 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:00.249943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e20b76f3-e9c0-4e89-ac77-07f33cea8044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:00.371972 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:00.376316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 679/1348] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:16:00.414717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a974d10-f8ca-4eac-beef-c9da4e157251 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:00.416317 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0a974d10-f8ca-4eac-beef-c9da4e157251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:00.524927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a974d10-f8ca-4eac-beef-c9da4e157251 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:00.576259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:00.604683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e21ef2-ea10-4fe4-86c4-79783ee9650e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:00.605345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e21ef2-ea10-4fe4-86c4-79783ee9650e service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:00.605637 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e21ef2-ea10-4fe4-86c4-79783ee9650e service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:00.605913 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e21ef2-ea10-4fe4-86c4-79783ee9650e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:00.607616 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-22e21ef2-ea10-4fe4-86c4-79783ee9650e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:00.725093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e21ef2-ea10-4fe4-86c4-79783ee9650e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:00.726548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e21ef2-ea10-4fe4-86c4-79783ee9650e service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:00.745584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 670/1349] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:05.704158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd5b8a-a2af-4ffa-ab2f-44a33f0f975c None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:05.704705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd5b8a-a2af-4ffa-ab2f-44a33f0f975c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:05.704912 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd5b8a-a2af-4ffa-ab2f-44a33f0f975c None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:05.706525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 680/1350] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jul 20 19:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:05.711549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3026d1-c13a-4b23-a646-97335092f9e6 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:05.712042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3026d1-c13a-4b23-a646-97335092f9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:05.712337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3026d1-c13a-4b23-a646-97335092f9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:05.842351 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-3d3026d1-c13a-4b23-a646-97335092f9e6 None None] MFA Rules not processed for user `ae581d3dda354e07894cb5205d1b2c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:05.843785 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-3d3026d1-c13a-4b23-a646-97335092f9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:05.862819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:05.966346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 671/1351] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jul 20 19:16:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:10.280150 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84a39b-d8b5-491c-b928-102ab00c4fb7 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:10.281100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84a39b-d8b5-491c-b928-102ab00c4fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:10.281551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84a39b-d8b5-491c-b928-102ab00c4fb7 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:10.283451 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 681/1352] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jul 20 19:16:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:10.288572 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b56614-8c46-416e-99f9-de832dadacfe None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:10.289333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b56614-8c46-416e-99f9-de832dadacfe None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:10.289773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b56614-8c46-416e-99f9-de832dadacfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:10.371639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-41b56614-8c46-416e-99f9-de832dadacfe None None] MFA Rules not processed for user `ae581d3dda354e07894cb5205d1b2c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:10.373166 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-41b56614-8c46-416e-99f9-de832dadacfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:10.451669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:10.496480 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 672/1353] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jul 20 19:16:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:11.719293 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcdc8f-4a9c-4aa8-bbeb-c6b72e76c831 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:11.719876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcdc8f-4a9c-4aa8-bbeb-c6b72e76c831 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:11.719876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcdc8f-4a9c-4aa8-bbeb-c6b72e76c831 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:11.721285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 682/1354] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jul 20 19:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:11.726087 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1add66-d85a-4204-a8f4-b64b6e56803b None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:11.726522 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1add66-d85a-4204-a8f4-b64b6e56803b None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:11.726636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1add66-d85a-4204-a8f4-b64b6e56803b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:11.803862 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-7f1add66-d85a-4204-a8f4-b64b6e56803b None None] MFA Rules not processed for user `ae581d3dda354e07894cb5205d1b2c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:11.805135 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-7f1add66-d85a-4204-a8f4-b64b6e56803b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:11.912751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 673/1355] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jul 20 19:16:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:15.294051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73115bbe-c036-41fd-8277-4bdc2457cd4b None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:15.294889 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73115bbe-c036-41fd-8277-4bdc2457cd4b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:15.295163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73115bbe-c036-41fd-8277-4bdc2457cd4b None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:15.297086 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 683/1356] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:16:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:15.303236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77277bf-ebfa-4a24-9903-2e5aa1df34e3 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:15.304115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77277bf-ebfa-4a24-9903-2e5aa1df34e3 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:15.304628 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77277bf-ebfa-4a24-9903-2e5aa1df34e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:15.358742 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:15.396114 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-a77277bf-ebfa-4a24-9903-2e5aa1df34e3 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:15.397559 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-a77277bf-ebfa-4a24-9903-2e5aa1df34e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:15.499011 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 674/1357] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:15.553540 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d532a-b2e0-47ae-a9be-865a7f27dc90 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:15.554310 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d532a-b2e0-47ae-a9be-865a7f27dc90 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:15.555254 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d532a-b2e0-47ae-a9be-865a7f27dc90 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:15.557248 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 684/1358] 10.222.0.43 () {54 vars in 962 bytes} [Wed Jul 20 19:16:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:15.565463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef7435-0090-42b3-ac68-3eb3b1d8c8af None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:15.565759 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef7435-0090-42b3-ac68-3eb3b1d8c8af None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:15.566039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef7435-0090-42b3-ac68-3eb3b1d8c8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:15.652236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-d8ef7435-0090-42b3-ac68-3eb3b1d8c8af None None] MFA Rules not processed for user `267b178236e046e4bc694b2da6f86eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:15.653363 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d8ef7435-0090-42b3-ac68-3eb3b1d8c8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:15.705236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:15.777218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 675/1359] 10.222.0.43 () {58 vars in 1059 bytes} [Wed Jul 20 19:16:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:15.784774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d532a-b2e0-47ae-a9be-865a7f27dc90 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:15.786445 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a71d532a-b2e0-47ae-a9be-865a7f27dc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:15.943600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d532a-b2e0-47ae-a9be-865a7f27dc90 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:16.031417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:16.077440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5f944-a0f1-410c-a8cc-e68eeef5cc25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:16.078084 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5f944-a0f1-410c-a8cc-e68eeef5cc25 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:16.078365 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5f944-a0f1-410c-a8cc-e68eeef5cc25 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:16.078690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5f944-a0f1-410c-a8cc-e68eeef5cc25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:16.080388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-39a5f944-a0f1-410c-a8cc-e68eeef5cc25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:16.186558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5f944-a0f1-410c-a8cc-e68eeef5cc25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:16.189546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5f944-a0f1-410c-a8cc-e68eeef5cc25 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:16.209482 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 685/1360] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:18.486769 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4252d68-3aca-4cf7-b3d1-1e2d186fe594 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:18.487271 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4252d68-3aca-4cf7-b3d1-1e2d186fe594 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:18.487271 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4252d68-3aca-4cf7-b3d1-1e2d186fe594 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:18.488643 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 676/1361] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:16:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:18.494113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805022ca-3121-4fe0-bcfb-2cdc5d306332 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:18.494535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805022ca-3121-4fe0-bcfb-2cdc5d306332 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:18.494741 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805022ca-3121-4fe0-bcfb-2cdc5d306332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:18.565272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-805022ca-3121-4fe0-bcfb-2cdc5d306332 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:18.566466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-805022ca-3121-4fe0-bcfb-2cdc5d306332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:18.605844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:18.678839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 686/1362] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:18.725146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4252d68-3aca-4cf7-b3d1-1e2d186fe594 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:18.726667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-a4252d68-3aca-4cf7-b3d1-1e2d186fe594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:18.899371 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4252d68-3aca-4cf7-b3d1-1e2d186fe594 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:18.909935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:19.030948 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39635536-1cb8-472d-97a2-d8fb5823a56b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:19.031605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39635536-1cb8-472d-97a2-d8fb5823a56b service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:19.031890 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39635536-1cb8-472d-97a2-d8fb5823a56b service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:19.032168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39635536-1cb8-472d-97a2-d8fb5823a56b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:19.034432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-39635536-1cb8-472d-97a2-d8fb5823a56b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:19.135191 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39635536-1cb8-472d-97a2-d8fb5823a56b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:19.136922 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39635536-1cb8-472d-97a2-d8fb5823a56b service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:19.158321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 677/1363] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:21.574424 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa2cd7-2ebd-4d51-ad17-712b29807c73 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:21.575594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa2cd7-2ebd-4d51-ad17-712b29807c73 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:21.575900 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa2cd7-2ebd-4d51-ad17-712b29807c73 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:21.577633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 687/1364] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:21.583640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b965ae-0857-4ce0-a8f7-da0ab9801263 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:21.584110 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b965ae-0857-4ce0-a8f7-da0ab9801263 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:21.584218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b965ae-0857-4ce0-a8f7-da0ab9801263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:21.599482 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:21.665806 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-31b965ae-0857-4ce0-a8f7-da0ab9801263 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:21.667486 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-31b965ae-0857-4ce0-a8f7-da0ab9801263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:21.774172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 678/1365] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:21.820945 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc745a5-0fae-417f-9e96-ef7a59f19bde None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:21.821232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc745a5-0fae-417f-9e96-ef7a59f19bde None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:21.821538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc745a5-0fae-417f-9e96-ef7a59f19bde None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:21.822927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 688/1366] 10.222.0.43 () {54 vars in 962 bytes} [Wed Jul 20 19:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:21.828417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382e532-2998-4871-a66c-0cc80223aba5 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:21.828655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382e532-2998-4871-a66c-0cc80223aba5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:21.828932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382e532-2998-4871-a66c-0cc80223aba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:21.912548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-0382e532-2998-4871-a66c-0cc80223aba5 None None] MFA Rules not processed for user `267b178236e046e4bc694b2da6f86eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:21.913761 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0382e532-2998-4871-a66c-0cc80223aba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:21.951954 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:22.048211 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 679/1367] 10.222.0.43 () {58 vars in 1059 bytes} [Wed Jul 20 19:16:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:22.056877 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc745a5-0fae-417f-9e96-ef7a59f19bde None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:22.058745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0bc745a5-0fae-417f-9e96-ef7a59f19bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:22.231354 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc745a5-0fae-417f-9e96-ef7a59f19bde None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:22.253747 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:22.361479 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab17b3-b591-4e05-b024-6285b5a436a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:22.362134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab17b3-b591-4e05-b024-6285b5a436a4 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:22.362415 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab17b3-b591-4e05-b024-6285b5a436a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:22.362685 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab17b3-b591-4e05-b024-6285b5a436a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:22.364376 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-66ab17b3-b591-4e05-b024-6285b5a436a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:22.470985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab17b3-b591-4e05-b024-6285b5a436a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:22.472665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab17b3-b591-4e05-b024-6285b5a436a4 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:22.493492 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 689/1368] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:23.515985 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6241976-dbb1-45bf-8d12-b79041bf9a72 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:23.516587 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6241976-dbb1-45bf-8d12-b79041bf9a72 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:23.517314 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6241976-dbb1-45bf-8d12-b79041bf9a72 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:23.519223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 680/1369] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:16:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:23.525560 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6528b5-76d2-42bc-8ae8-92b556cba022 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:23.526063 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6528b5-76d2-42bc-8ae8-92b556cba022 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:23.526144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6528b5-76d2-42bc-8ae8-92b556cba022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:23.569636 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:23.612767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ba6528b5-76d2-42bc-8ae8-92b556cba022 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:23.614030 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ba6528b5-76d2-42bc-8ae8-92b556cba022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:23.759791 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 690/1370] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:24.104150 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cae064-55d2-425e-823b-33542618dfe6 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:24.104150 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cae064-55d2-425e-823b-33542618dfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:24.104150 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cae064-55d2-425e-823b-33542618dfe6 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:24.107415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 681/1371] 10.222.0.43 () {54 vars in 964 bytes} [Wed Jul 20 19:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:24.110340 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960ed85-bdf8-4bda-933b-ab1e13290f28 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:24.110708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960ed85-bdf8-4bda-933b-ab1e13290f28 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:24.110975 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960ed85-bdf8-4bda-933b-ab1e13290f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:24.189194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2960ed85-bdf8-4bda-933b-ab1e13290f28 None None] MFA Rules not processed for user `69aef294bb4943c3822ad7d0a3d91fd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:24.190413 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2960ed85-bdf8-4bda-933b-ab1e13290f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:24.240560 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:24.272977 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 691/1372] 10.222.0.43 () {58 vars in 1061 bytes} [Wed Jul 20 19:16:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:24.281113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cae064-55d2-425e-823b-33542618dfe6 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:24.282780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-77cae064-55d2-425e-823b-33542618dfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:24.408150 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cae064-55d2-425e-823b-33542618dfe6 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:24.452823 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:24.498246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077d875-de95-4e18-9a18-a60baf84df56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:24.499394 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077d875-de95-4e18-9a18-a60baf84df56 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:24.499394 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077d875-de95-4e18-9a18-a60baf84df56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:24.499809 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077d875-de95-4e18-9a18-a60baf84df56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:24.502648 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-b077d875-de95-4e18-9a18-a60baf84df56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:24.626760 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077d875-de95-4e18-9a18-a60baf84df56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:24.628517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077d875-de95-4e18-9a18-a60baf84df56 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:24.647166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110314f-c6f2-4ccd-ab04-750c94ea875d None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:24.647592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110314f-c6f2-4ccd-ab04-750c94ea875d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:24.647686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 682/1373] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:24.647974 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110314f-c6f2-4ccd-ab04-750c94ea875d None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:24.649891 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 692/1374] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:16:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:16:24.655231 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd297f84-460f-4085-a604-1919fe25b07a None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:24.655679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd297f84-460f-4085-a604-1919fe25b07a None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:24.655972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd297f84-460f-4085-a604-1919fe25b07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:24.738000 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-fd297f84-460f-4085-a604-1919fe25b07a None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:24.739613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-fd297f84-460f-4085-a604-1919fe25b07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:24.769117 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:24.885167 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 683/1375] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:24.956115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110314f-c6f2-4ccd-ab04-750c94ea875d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:24.957695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4110314f-c6f2-4ccd-ab04-750c94ea875d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:25.106575 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110314f-c6f2-4ccd-ab04-750c94ea875d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:25.160749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:25.223878 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda043e-2199-453c-bc92-829b8ec8fe68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:25.224552 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda043e-2199-453c-bc92-829b8ec8fe68 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:25.224838 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda043e-2199-453c-bc92-829b8ec8fe68 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:25.225142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda043e-2199-453c-bc92-829b8ec8fe68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:25.226817 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2eda043e-2199-453c-bc92-829b8ec8fe68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:25.317679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda043e-2199-453c-bc92-829b8ec8fe68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:25.319324 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda043e-2199-453c-bc92-829b8ec8fe68 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:25.339606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 693/1376] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:27.391988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d561c-80e7-420d-a4c3-6b7d8962b167 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:27.392503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d561c-80e7-420d-a4c3-6b7d8962b167 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:27.392503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d561c-80e7-420d-a4c3-6b7d8962b167 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:27.393858 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 684/1377] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:27.398940 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53ae6c-5125-41df-b4b3-dc46342b7e4a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:27.399348 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53ae6c-5125-41df-b4b3-dc46342b7e4a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:27.399507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53ae6c-5125-41df-b4b3-dc46342b7e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:27.464430 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2f53ae6c-5125-41df-b4b3-dc46342b7e4a None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:27.465627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2f53ae6c-5125-41df-b4b3-dc46342b7e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:27.472795 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:27.573071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 694/1378] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:27.617909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d561c-80e7-420d-a4c3-6b7d8962b167 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:27.619477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-eb0d561c-80e7-420d-a4c3-6b7d8962b167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:27.794472 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d561c-80e7-420d-a4c3-6b7d8962b167 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:27.798662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:27.914200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e3159-8062-47ae-91db-2cf792ae3b20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:27.914761 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e3159-8062-47ae-91db-2cf792ae3b20 service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:27.915050 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e3159-8062-47ae-91db-2cf792ae3b20 service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:27.915324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e3159-8062-47ae-91db-2cf792ae3b20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:27.917070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-139e3159-8062-47ae-91db-2cf792ae3b20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:28.019094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e3159-8062-47ae-91db-2cf792ae3b20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:28.020733 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e3159-8062-47ae-91db-2cf792ae3b20 service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:28.039490 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 685/1379] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:30.172443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347af68-530e-49a2-917f-7c2e4d6d47e6 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:30.173026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347af68-530e-49a2-917f-7c2e4d6d47e6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:30.173113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347af68-530e-49a2-917f-7c2e4d6d47e6 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:30.175254 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 695/1380] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:30.180723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6b546-de32-4d33-ae27-11bd86b631ea None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:30.181206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6b546-de32-4d33-ae27-11bd86b631ea None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:30.181286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6b546-de32-4d33-ae27-11bd86b631ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:30.192417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:30.252253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-fcc6b546-de32-4d33-ae27-11bd86b631ea None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:30.253488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-fcc6b546-de32-4d33-ae27-11bd86b631ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:30.379471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 686/1381] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:30.424458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347af68-530e-49a2-917f-7c2e4d6d47e6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:30.426103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e347af68-530e-49a2-917f-7c2e4d6d47e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:30.576486 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:30.588389 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347af68-530e-49a2-917f-7c2e4d6d47e6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:30.715997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4985b-8c9d-4eff-a018-2a76c47ee1ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:30.716764 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4985b-8c9d-4eff-a018-2a76c47ee1ae service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:30.717114 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4985b-8c9d-4eff-a018-2a76c47ee1ae service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:30.717458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4985b-8c9d-4eff-a018-2a76c47ee1ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:30.719176 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b1f4985b-8c9d-4eff-a018-2a76c47ee1ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:30.813176 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4985b-8c9d-4eff-a018-2a76c47ee1ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:30.815415 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4985b-8c9d-4eff-a018-2a76c47ee1ae service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:30.826445 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:30.836327 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 696/1382] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:32.932846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978942f3-fda3-4224-a79a-a890d5130a36 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:32.933862 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978942f3-fda3-4224-a79a-a890d5130a36 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:32.933862 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978942f3-fda3-4224-a79a-a890d5130a36 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:32.935094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 687/1383] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:32.941230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d7ccf-afb5-47f8-b386-fe4f4d370b71 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:32.941688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d7ccf-afb5-47f8-b386-fe4f4d370b71 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:32.942022 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d7ccf-afb5-47f8-b386-fe4f4d370b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:33.027321 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2b5d7ccf-afb5-47f8-b386-fe4f4d370b71 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:33.028964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2b5d7ccf-afb5-47f8-b386-fe4f4d370b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:33.150740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 697/1384] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:33.196812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978942f3-fda3-4224-a79a-a890d5130a36 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:33.198272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-978942f3-fda3-4224-a79a-a890d5130a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:33.296737 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:33.367576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978942f3-fda3-4224-a79a-a890d5130a36 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:33.494531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ccca5-6c73-465f-9ae5-e16a6afba3e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:33.495154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ccca5-6c73-465f-9ae5-e16a6afba3e2 service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:33.495465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ccca5-6c73-465f-9ae5-e16a6afba3e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:33.495746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ccca5-6c73-465f-9ae5-e16a6afba3e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:33.497576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-7d6ccca5-6c73-465f-9ae5-e16a6afba3e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:33.554332 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:33.599240 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ccca5-6c73-465f-9ae5-e16a6afba3e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:33.600847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ccca5-6c73-465f-9ae5-e16a6afba3e2 service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:33.620002 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 688/1385] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:35.752145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500e471-f234-4a87-ac18-b403f005ab31 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:35.752727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500e471-f234-4a87-ac18-b403f005ab31 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:35.752885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500e471-f234-4a87-ac18-b403f005ab31 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:35.754679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 698/1386] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:16:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:35.761411 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e516a6e-4733-4951-a716-2c5c14c85799 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:35.761867 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e516a6e-4733-4951-a716-2c5c14c85799 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:35.761973 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e516a6e-4733-4951-a716-2c5c14c85799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:35.835623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-7e516a6e-4733-4951-a716-2c5c14c85799 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:35.837316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-7e516a6e-4733-4951-a716-2c5c14c85799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:35.942650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 689/1387] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:35.989258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500e471-f234-4a87-ac18-b403f005ab31 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:35.990808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f500e471-f234-4a87-ac18-b403f005ab31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:36.071118 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:36.143116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500e471-f234-4a87-ac18-b403f005ab31 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:36.263942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418bb7c-c9c8-48f7-a5bc-af066a0a9251 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:36.264629 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418bb7c-c9c8-48f7-a5bc-af066a0a9251 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:36.264913 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418bb7c-c9c8-48f7-a5bc-af066a0a9251 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:36.265190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418bb7c-c9c8-48f7-a5bc-af066a0a9251 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:36.266925 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3418bb7c-c9c8-48f7-a5bc-af066a0a9251 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:36.310980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:36.375154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418bb7c-c9c8-48f7-a5bc-af066a0a9251 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:36.377423 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418bb7c-c9c8-48f7-a5bc-af066a0a9251 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:36.396300 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 699/1388] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:38.452606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d289b6-e3eb-4384-ad03-4e39cdace2ec None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:38.453214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d289b6-e3eb-4384-ad03-4e39cdace2ec None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:38.453311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d289b6-e3eb-4384-ad03-4e39cdace2ec None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:38.454625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 690/1389] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:38.459993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f92c00-26aa-4f51-8f1c-af45a3928914 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:38.460445 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f92c00-26aa-4f51-8f1c-af45a3928914 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:38.460593 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f92c00-26aa-4f51-8f1c-af45a3928914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:38.527206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a9f92c00-26aa-4f51-8f1c-af45a3928914 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:38.528425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a9f92c00-26aa-4f51-8f1c-af45a3928914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:38.627043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 700/1390] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:38.671719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d289b6-e3eb-4384-ad03-4e39cdace2ec None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:38.673426 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-18d289b6-e3eb-4384-ad03-4e39cdace2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:38.704654 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:38.819479 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d289b6-e3eb-4384-ad03-4e39cdace2ec None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:38.919331 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:38.931226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2992de-a59d-42a5-9c27-1e41251da642 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:38.931899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2992de-a59d-42a5-9c27-1e41251da642 service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:38.932176 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2992de-a59d-42a5-9c27-1e41251da642 service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:38.932562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2992de-a59d-42a5-9c27-1e41251da642 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:38.934478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-fb2992de-a59d-42a5-9c27-1e41251da642 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:39.025202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2992de-a59d-42a5-9c27-1e41251da642 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:39.026800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2992de-a59d-42a5-9c27-1e41251da642 service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:39.045612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 691/1391] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:39.987985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78b48e-daf8-4068-8186-36dc344e77a1 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:39.988601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78b48e-daf8-4068-8186-36dc344e77a1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:39.988601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78b48e-daf8-4068-8186-36dc344e77a1 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:39.989965 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 701/1392] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:39.995313 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa087e-83a8-46fd-b253-e0d92f1954b4 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:39.995751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa087e-83a8-46fd-b253-e0d92f1954b4 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:39.995874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa087e-83a8-46fd-b253-e0d92f1954b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:40.059713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4faa087e-83a8-46fd-b253-e0d92f1954b4 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:40.060932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4faa087e-83a8-46fd-b253-e0d92f1954b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:40.119982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:40.158803 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 692/1393] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:40.214477 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006d6f9-520c-45cb-92ca-104e137964a6 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:40.214764 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006d6f9-520c-45cb-92ca-104e137964a6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:40.215033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006d6f9-520c-45cb-92ca-104e137964a6 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:40.216423 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 702/1394] 10.222.0.43 () {54 vars in 966 bytes} [Wed Jul 20 19:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:40.222059 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c4795-34df-4e6e-900c-72df491aa092 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:40.222414 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c4795-34df-4e6e-900c-72df491aa092 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:40.222756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c4795-34df-4e6e-900c-72df491aa092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:40.296529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-212c4795-34df-4e6e-900c-72df491aa092 None None] MFA Rules not processed for user `d3e28557896e41c893f0de2590bc96d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:40.297611 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-212c4795-34df-4e6e-900c-72df491aa092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:40.388035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 693/1395] 10.222.0.43 () {58 vars in 1063 bytes} [Wed Jul 20 19:16:40 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:40.396227 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006d6f9-520c-45cb-92ca-104e137964a6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:40.397799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7006d6f9-520c-45cb-92ca-104e137964a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:40.412495 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:40.551588 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006d6f9-520c-45cb-92ca-104e137964a6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:40.627028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f999e72-4eb3-4d68-afb9-4e9f1ff677c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3e28557896e41c893f0de2590bc96d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:40.627672 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f999e72-4eb3-4d68-afb9-4e9f1ff677c2 service glance] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:40.627952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f999e72-4eb3-4d68-afb9-4e9f1ff677c2 service glance] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:40.628225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f999e72-4eb3-4d68-afb9-4e9f1ff677c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:40.629894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5f999e72-4eb3-4d68-afb9-4e9f1ff677c2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:40.699340 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:40.742658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f999e72-4eb3-4d68-afb9-4e9f1ff677c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:40.743776 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f999e72-4eb3-4d68-afb9-4e9f1ff677c2 service glance] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:40.762595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 703/1396] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:41.167328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f9e10-d618-4314-8562-31894284d728 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:41.167882 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f9e10-d618-4314-8562-31894284d728 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:41.167882 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f9e10-d618-4314-8562-31894284d728 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:41.169314 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 694/1397] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:41.175153 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d866cb2-1d91-4a1c-a8d8-4173138d824a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:41.175589 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d866cb2-1d91-4a1c-a8d8-4173138d824a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:41.175708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d866cb2-1d91-4a1c-a8d8-4173138d824a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:41.243164 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-6d866cb2-1d91-4a1c-a8d8-4173138d824a None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:41.244424 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6d866cb2-1d91-4a1c-a8d8-4173138d824a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:41.349738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 704/1398] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:41.397451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f9e10-d618-4314-8562-31894284d728 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:41.398883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ef2f9e10-d618-4314-8562-31894284d728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:41.474169 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0374d-0020-43aa-bc26-4a340625667b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:41.474490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0374d-0020-43aa-bc26-4a340625667b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:41.474774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0374d-0020-43aa-bc26-4a340625667b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:41.480126 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:41.500151 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:41.541765 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-6ad0374d-0020-43aa-bc26-4a340625667b None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:41.542995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6ad0374d-0020-43aa-bc26-4a340625667b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:41.570282 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f9e10-d618-4314-8562-31894284d728 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:41.631677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 705/1399] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jul 20 19:16:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:16:41.678593 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d55f49-fec9-4023-86f3-fcf3d09085b4 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:41.678886 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d55f49-fec9-4023-86f3-fcf3d09085b4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:41.679170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d55f49-fec9-4023-86f3-fcf3d09085b4 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:41.680612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 706/1400] 10.222.0.43 () {54 vars in 943 bytes} [Wed Jul 20 19:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:16:41.685473 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe6fc3-340d-42bc-83f0-c5ec1bd56e6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:41.686128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe6fc3-340d-42bc-83f0-c5ec1bd56e6f service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:41.686411 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe6fc3-340d-42bc-83f0-c5ec1bd56e6f service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:41.686686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe6fc3-340d-42bc-83f0-c5ec1bd56e6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:41.686794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c79be-db56-4c53-957a-279ed611abb4 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:41.687034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c79be-db56-4c53-957a-279ed611abb4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:41.687316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c79be-db56-4c53-957a-279ed611abb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:41.688452 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-cdfe6fc3-340d-42bc-83f0-c5ec1bd56e6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:41.710689 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:41.764403 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7e1c79be-db56-4c53-957a-279ed611abb4 None None] MFA Rules not processed for user `1c944eab8a8948749bb8375787216b09`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:41.765615 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7e1c79be-db56-4c53-957a-279ed611abb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:41.780239 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe6fc3-340d-42bc-83f0-c5ec1bd56e6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:41.781846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe6fc3-340d-42bc-83f0-c5ec1bd56e6f service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:41.808420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 695/1401] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:16:41.825443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:41.886478 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 707/1402] 10.222.0.43 () {58 vars in 1040 bytes} [Wed Jul 20 19:16:41 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:16:41.896241 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe6fc3-340d-42bc-83f0-c5ec1bd56e6f service nova] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:41.897520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-cdfe6fc3-340d-42bc-83f0-c5ec1bd56e6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:41.998939 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe6fc3-340d-42bc-83f0-c5ec1bd56e6f service nova] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:42.071165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:42.124518 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4accbeb-bce6-4694-a471-c4c6bb3d8e86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:42.125180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4accbeb-bce6-4694-a471-c4c6bb3d8e86 service swift] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:42.125517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4accbeb-bce6-4694-a471-c4c6bb3d8e86 service swift] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:42.125786 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4accbeb-bce6-4694-a471-c4c6bb3d8e86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:42.127510 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-a4accbeb-bce6-4694-a471-c4c6bb3d8e86 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:42.219516 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4accbeb-bce6-4694-a471-c4c6bb3d8e86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:42.221674 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4accbeb-bce6-4694-a471-c4c6bb3d8e86 service swift] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:42.231039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 696/1403] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:16:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:16:43.881555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390a256-a758-45df-8f3c-a37ceee9de7f None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:43.882124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390a256-a758-45df-8f3c-a37ceee9de7f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:43.882124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390a256-a758-45df-8f3c-a37ceee9de7f None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:43.883491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 708/1404] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:43.888721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b6c94-9491-4d55-99a4-9bedbfedf0b1 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:43.889140 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b6c94-9491-4d55-99a4-9bedbfedf0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:43.889316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b6c94-9491-4d55-99a4-9bedbfedf0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:43.956344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-398b6c94-9491-4d55-99a4-9bedbfedf0b1 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:43.957714 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-398b6c94-9491-4d55-99a4-9bedbfedf0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:44.005743 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:44.067953 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 697/1405] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:44.112788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390a256-a758-45df-8f3c-a37ceee9de7f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:44.114327 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b390a256-a758-45df-8f3c-a37ceee9de7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:44.272510 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390a256-a758-45df-8f3c-a37ceee9de7f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:44.301773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:44.403614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d49850-a87e-4a90-8a74-a03d681e4bdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:44.404332 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d49850-a87e-4a90-8a74-a03d681e4bdb service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:44.404610 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d49850-a87e-4a90-8a74-a03d681e4bdb service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:44.404890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d49850-a87e-4a90-8a74-a03d681e4bdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:44.406574 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-59d49850-a87e-4a90-8a74-a03d681e4bdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:44.511635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d49850-a87e-4a90-8a74-a03d681e4bdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:44.513283 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d49850-a87e-4a90-8a74-a03d681e4bdb service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:44.532782 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 709/1406] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:46.638097 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b7d83-525b-4d1b-a6c9-fd4921683b59 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:46.638641 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b7d83-525b-4d1b-a6c9-fd4921683b59 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:46.638641 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b7d83-525b-4d1b-a6c9-fd4921683b59 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:46.639939 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 698/1407] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:46.646525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5031a75-3687-4b8d-8dc8-d64d9010da62 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:46.647001 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5031a75-3687-4b8d-8dc8-d64d9010da62 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:46.647083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5031a75-3687-4b8d-8dc8-d64d9010da62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:46.699089 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:46.727929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f5031a75-3687-4b8d-8dc8-d64d9010da62 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:46.729280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f5031a75-3687-4b8d-8dc8-d64d9010da62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:46.843481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 710/1408] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:46.888997 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b7d83-525b-4d1b-a6c9-fd4921683b59 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:46.890529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d63b7d83-525b-4d1b-a6c9-fd4921683b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:47.051494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b7d83-525b-4d1b-a6c9-fd4921683b59 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:47.080387 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:47.176576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ff2ab-6c97-406f-894d-0eea82026ac8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:47.177495 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ff2ab-6c97-406f-894d-0eea82026ac8 service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:47.177780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ff2ab-6c97-406f-894d-0eea82026ac8 service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:47.178051 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ff2ab-6c97-406f-894d-0eea82026ac8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:47.179710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-aa7ff2ab-6c97-406f-894d-0eea82026ac8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:47.274290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ff2ab-6c97-406f-894d-0eea82026ac8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:47.278108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ff2ab-6c97-406f-894d-0eea82026ac8 service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:47.302061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 699/1409] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:56.839687 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=51 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:57.299482 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=86 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:04.641095 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8083e68-556f-460c-bc5c-4551c5a715bc None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:04.642422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8083e68-556f-460c-bc5c-4551c5a715bc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:04.642422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8083e68-556f-460c-bc5c-4551c5a715bc None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:04.643693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 711/1410] 10.222.0.43 () {54 vars in 918 bytes} [Wed Jul 20 19:17:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:17:04.648855 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5431952-0b69-4b6d-9589-0f53c7a5cb0c None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:04.649271 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5431952-0b69-4b6d-9589-0f53c7a5cb0c None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:04.649433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5431952-0b69-4b6d-9589-0f53c7a5cb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:04.723736 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-d5431952-0b69-4b6d-9589-0f53c7a5cb0c None None] MFA Rules not processed for user `ae581d3dda354e07894cb5205d1b2c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:04.724964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d5431952-0b69-4b6d-9589-0f53c7a5cb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:04.829462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 700/1411] 10.222.0.43 () {58 vars in 1015 bytes} [Wed Jul 20 19:17:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:17:04.843304 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0846c4-ef33-41a9-b0f4-de516f98c6b9 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:04.843598 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0846c4-ef33-41a9-b0f4-de516f98c6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:04.843879 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0846c4-ef33-41a9-b0f4-de516f98c6b9 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:04.845922 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 712/1412] 10.222.0.43 () {54 vars in 918 bytes} [Wed Jul 20 19:17:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:04.851169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9144b3-ba97-469d-94a1-de9c5e0cffa8 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:04.851463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9144b3-ba97-469d-94a1-de9c5e0cffa8 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:04.851744 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9144b3-ba97-469d-94a1-de9c5e0cffa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:04.921627 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:04.924211 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-2c9144b3-ba97-469d-94a1-de9c5e0cffa8 None None] MFA Rules not processed for user `267b178236e046e4bc694b2da6f86eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:04.926269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-2c9144b3-ba97-469d-94a1-de9c5e0cffa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:05.040982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 701/1413] 10.222.0.43 () {58 vars in 1015 bytes} [Wed Jul 20 19:17:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:17:07.551380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085e67a-1b25-4db1-8318-ba37f1801288 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:07.551953 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085e67a-1b25-4db1-8318-ba37f1801288 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:07.551953 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085e67a-1b25-4db1-8318-ba37f1801288 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:07.553278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:07.562751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d098cbd-44f6-4186-ba10-998b08d856a5 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:07.563777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d098cbd-44f6-4186-ba10-998b08d856a5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:07.563889 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d098cbd-44f6-4186-ba10-998b08d856a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:07.633925 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-6d098cbd-44f6-4186-ba10-998b08d856a5 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:07.635132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6d098cbd-44f6-4186-ba10-998b08d856a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:07.689173 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:07.731676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:07.775558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085e67a-1b25-4db1-8318-ba37f1801288 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:07.777372 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5085e67a-1b25-4db1-8318-ba37f1801288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:07.926095 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085e67a-1b25-4db1-8318-ba37f1801288 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:07.999902 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:08.039940 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10dd9c-56ba-49d6-a250-ea2f70c9d994 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:08.040617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10dd9c-56ba-49d6-a250-ea2f70c9d994 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:08.040901 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10dd9c-56ba-49d6-a250-ea2f70c9d994 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:08.041173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10dd9c-56ba-49d6-a250-ea2f70c9d994 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:08.042833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9e10dd9c-56ba-49d6-a250-ea2f70c9d994 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:08.135588 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10dd9c-56ba-49d6-a250-ea2f70c9d994 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:08.137209 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10dd9c-56ba-49d6-a250-ea2f70c9d994 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:08.155052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 714/1416] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:09.058764 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926d52e-948d-44da-9a06-69183c576499 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:09.059338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926d52e-948d-44da-9a06-69183c576499 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:09.059338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926d52e-948d-44da-9a06-69183c576499 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:09.060746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:09.066147 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822be5d-8989-4c02-89e5-3823a2b4d68b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:09.066566 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822be5d-8989-4c02-89e5-3823a2b4d68b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:09.066715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822be5d-8989-4c02-89e5-3823a2b4d68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:09.142616 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d822be5d-8989-4c02-89e5-3823a2b4d68b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:09.143831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d822be5d-8989-4c02-89e5-3823a2b4d68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:09.171267 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:09.243456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:17:09.393428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926d52e-948d-44da-9a06-69183c576499 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:09.394974 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e926d52e-948d-44da-9a06-69183c576499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:09.539149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926d52e-948d-44da-9a06-69183c576499 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:09.578258 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:09.652856 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694eb4b1-2532-4e78-a8b4-71eb3692331a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:09.653430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694eb4b1-2532-4e78-a8b4-71eb3692331a service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:09.653718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694eb4b1-2532-4e78-a8b4-71eb3692331a service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:09.653996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694eb4b1-2532-4e78-a8b4-71eb3692331a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:09.655667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-694eb4b1-2532-4e78-a8b4-71eb3692331a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:09.746887 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694eb4b1-2532-4e78-a8b4-71eb3692331a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:09.748536 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694eb4b1-2532-4e78-a8b4-71eb3692331a service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:09.766440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 704/1419] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:10.658991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa1891-8f05-4fdf-ade9-d75f14193d56 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:10.659519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa1891-8f05-4fdf-ade9-d75f14193d56 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:10.659519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa1891-8f05-4fdf-ade9-d75f14193d56 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:10.660885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:17:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:10.666356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedba03-3193-4722-824d-82a61a1ca926 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:10.666847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedba03-3193-4722-824d-82a61a1ca926 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:10.667306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedba03-3193-4722-824d-82a61a1ca926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:10.723544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:10.748617 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-6fedba03-3193-4722-824d-82a61a1ca926 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:10.749843 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6fedba03-3193-4722-824d-82a61a1ca926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:10.846427 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:11.002896 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa1891-8f05-4fdf-ade9-d75f14193d56 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:11.004983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9bfa1891-8f05-4fdf-ade9-d75f14193d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:11.151492 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa1891-8f05-4fdf-ade9-d75f14193d56 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:11.164429 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:11.269953 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c7475-0795-486e-a1a2-054b6ebc607b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:11.270642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c7475-0795-486e-a1a2-054b6ebc607b service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:11.270923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c7475-0795-486e-a1a2-054b6ebc607b service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:11.271195 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c7475-0795-486e-a1a2-054b6ebc607b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:11.272891 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f23c7475-0795-486e-a1a2-054b6ebc607b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:11.374060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c7475-0795-486e-a1a2-054b6ebc607b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:11.376340 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c7475-0795-486e-a1a2-054b6ebc607b service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:11.394956 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 717/1422] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:12.262694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c80eb3-675a-4210-82e1-35617b46342e None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:12.263321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c80eb3-675a-4210-82e1-35617b46342e None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:12.263439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c80eb3-675a-4210-82e1-35617b46342e None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:12.264827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:12.270697 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbe7ea-6696-40fc-922f-3f8cce3a3901 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:12.271152 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbe7ea-6696-40fc-922f-3f8cce3a3901 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:12.271249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbe7ea-6696-40fc-922f-3f8cce3a3901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:12.290467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:12.350645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-05cbe7ea-6696-40fc-922f-3f8cce3a3901 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:12.351902 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-05cbe7ea-6696-40fc-922f-3f8cce3a3901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:12.458012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:12.609008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c80eb3-675a-4210-82e1-35617b46342e None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:12.611333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-48c80eb3-675a-4210-82e1-35617b46342e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:12.723319 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:12.762642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c80eb3-675a-4210-82e1-35617b46342e None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:12.876946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bdcab-95f2-4d9c-ac77-43853815d152 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:12.877510 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bdcab-95f2-4d9c-ac77-43853815d152 service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:12.877797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bdcab-95f2-4d9c-ac77-43853815d152 service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:12.878088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bdcab-95f2-4d9c-ac77-43853815d152 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:12.879872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5a6bdcab-95f2-4d9c-ac77-43853815d152 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:12.960437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:12.975555 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bdcab-95f2-4d9c-ac77-43853815d152 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:12.977182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bdcab-95f2-4d9c-ac77-43853815d152 service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:12.996029 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 707/1425] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:13.906525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7153c6-5864-4d47-a45b-7204b6af869c None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:13.907140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7153c6-5864-4d47-a45b-7204b6af869c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:13.907260 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7153c6-5864-4d47-a45b-7204b6af869c None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:13.908884 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:17:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:13.914511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6fa87-a3e7-4ab8-b061-9f0748e9dc55 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:13.914977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6fa87-a3e7-4ab8-b061-9f0748e9dc55 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:13.915181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6fa87-a3e7-4ab8-b061-9f0748e9dc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:14.001347 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-65b6fa87-a3e7-4ab8-b061-9f0748e9dc55 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:14.002575 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-65b6fa87-a3e7-4ab8-b061-9f0748e9dc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:14.099235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:14.249884 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7153c6-5864-4d47-a45b-7204b6af869c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:14.251898 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2b7153c6-5864-4d47-a45b-7204b6af869c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:14.343867 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:14.406947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7153c6-5864-4d47-a45b-7204b6af869c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:14.521255 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18108fc9-090f-4867-ab62-e49976d710d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:14.521812 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18108fc9-090f-4867-ab62-e49976d710d6 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:14.522096 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18108fc9-090f-4867-ab62-e49976d710d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:14.522370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18108fc9-090f-4867-ab62-e49976d710d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:14.524041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-18108fc9-090f-4867-ab62-e49976d710d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:14.573493 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:14.615184 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18108fc9-090f-4867-ab62-e49976d710d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:14.616817 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18108fc9-090f-4867-ab62-e49976d710d6 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:14.635384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 720/1428] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:15.444251 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3d4b2-1695-4565-8946-f14acb75a9eb None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:15.444799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3d4b2-1695-4565-8946-f14acb75a9eb None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:15.444799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3d4b2-1695-4565-8946-f14acb75a9eb None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:15.446121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:15.451006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601d22d-3999-4817-8db6-8cfd26bd6a4b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:15.451608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601d22d-3999-4817-8db6-8cfd26bd6a4b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:15.451713 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601d22d-3999-4817-8db6-8cfd26bd6a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:15.527123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-6601d22d-3999-4817-8db6-8cfd26bd6a4b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:15.528371 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6601d22d-3999-4817-8db6-8cfd26bd6a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:15.625267 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:17:15.775668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3d4b2-1695-4565-8946-f14acb75a9eb None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:15.777144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-fcb3d4b2-1695-4565-8946-f14acb75a9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:15.825695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:15.921901 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3d4b2-1695-4565-8946-f14acb75a9eb None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:16.036919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c4210-57e0-461e-b560-fe0faf0aa0a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:16.037494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c4210-57e0-461e-b560-fe0faf0aa0a4 service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:16.037781 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c4210-57e0-461e-b560-fe0faf0aa0a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:16.038067 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c4210-57e0-461e-b560-fe0faf0aa0a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:16.039735 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-801c4210-57e0-461e-b560-fe0faf0aa0a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:16.046711 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:16.130775 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c4210-57e0-461e-b560-fe0faf0aa0a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:16.132406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c4210-57e0-461e-b560-fe0faf0aa0a4 service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:16.150346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 710/1431] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:16.962271 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280fdf2-2b35-4d29-9f9c-f6b714380e75 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:16.962863 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280fdf2-2b35-4d29-9f9c-f6b714380e75 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:16.962863 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280fdf2-2b35-4d29-9f9c-f6b714380e75 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:16.964269 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:16.969261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b63cf-bab7-42ba-aea0-b994326b3a85 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:16.969706 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b63cf-bab7-42ba-aea0-b994326b3a85 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:16.969803 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b63cf-bab7-42ba-aea0-b994326b3a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:17.045066 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-828b63cf-bab7-42ba-aea0-b994326b3a85 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:17.046304 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-828b63cf-bab7-42ba-aea0-b994326b3a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:17.116145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:17.144802 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:17.298646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280fdf2-2b35-4d29-9f9c-f6b714380e75 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:17.300225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c280fdf2-2b35-4d29-9f9c-f6b714380e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:17.318436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:17.449517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280fdf2-2b35-4d29-9f9c-f6b714380e75 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:17.537582 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:17.564447 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75d3f5-68ef-46ee-bc1c-45ad70892f6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:17.565099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75d3f5-68ef-46ee-bc1c-45ad70892f6b service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:17.565381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75d3f5-68ef-46ee-bc1c-45ad70892f6b service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:17.565653 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75d3f5-68ef-46ee-bc1c-45ad70892f6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:17.567323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-be75d3f5-68ef-46ee-bc1c-45ad70892f6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:17.655094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75d3f5-68ef-46ee-bc1c-45ad70892f6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:17.657330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75d3f5-68ef-46ee-bc1c-45ad70892f6b service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:17.675298 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 723/1434] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:18.485967 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab786def-9126-4d35-a8f9-58d84ffbed44 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:18.486501 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab786def-9126-4d35-a8f9-58d84ffbed44 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:18.486501 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab786def-9126-4d35-a8f9-58d84ffbed44 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:18.487813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:17:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:18.492813 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20263953-3cbd-4393-9ae0-067efd997ebb None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:18.493205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20263953-3cbd-4393-9ae0-067efd997ebb None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:18.493369 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20263953-3cbd-4393-9ae0-067efd997ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:18.566874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-20263953-3cbd-4393-9ae0-067efd997ebb None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:18.568058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-20263953-3cbd-4393-9ae0-067efd997ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:18.608505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:18.663216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:18.817847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab786def-9126-4d35-a8f9-58d84ffbed44 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:18.819359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ab786def-9126-4d35-a8f9-58d84ffbed44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:18.962369 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab786def-9126-4d35-a8f9-58d84ffbed44 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:19.015954 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:19.076264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5034e83-9ad6-4fc3-81e4-1e1e991c93f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:19.076890 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5034e83-9ad6-4fc3-81e4-1e1e991c93f4 service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:19.077177 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5034e83-9ad6-4fc3-81e4-1e1e991c93f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:19.077455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5034e83-9ad6-4fc3-81e4-1e1e991c93f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:19.079127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d5034e83-9ad6-4fc3-81e4-1e1e991c93f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:19.181592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5034e83-9ad6-4fc3-81e4-1e1e991c93f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:19.183178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5034e83-9ad6-4fc3-81e4-1e1e991c93f4 service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:19.201512 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 713/1437] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:17:20.029127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b00ee-ad37-492d-beb0-8e9a8f83b3e9 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:20.029650 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b00ee-ad37-492d-beb0-8e9a8f83b3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:20.029768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b00ee-ad37-492d-beb0-8e9a8f83b3e9 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:20.031054 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:20.036231 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272dfcb-9eee-4271-9b28-2c0400d311c6 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:20.036714 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272dfcb-9eee-4271-9b28-2c0400d311c6 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:20.036841 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272dfcb-9eee-4271-9b28-2c0400d311c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:20.115416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4272dfcb-9eee-4271-9b28-2c0400d311c6 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:20.116892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4272dfcb-9eee-4271-9b28-2c0400d311c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:20.118974 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:20.214067 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:20.369590 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b00ee-ad37-492d-beb0-8e9a8f83b3e9 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:20.371122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-991b00ee-ad37-492d-beb0-8e9a8f83b3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:20.525282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b00ee-ad37-492d-beb0-8e9a8f83b3e9 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:20.557770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:20.655922 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de95c9a-b861-4f68-b0c5-c8f60108e54d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:20.656511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de95c9a-b861-4f68-b0c5-c8f60108e54d service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:20.657337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de95c9a-b861-4f68-b0c5-c8f60108e54d service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:20.657615 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de95c9a-b861-4f68-b0c5-c8f60108e54d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:20.659321 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3de95c9a-b861-4f68-b0c5-c8f60108e54d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:20.750840 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de95c9a-b861-4f68-b0c5-c8f60108e54d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:20.752480 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de95c9a-b861-4f68-b0c5-c8f60108e54d service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:20.770760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 726/1440] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:22.715417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0278a-998b-455f-ace7-6f54ff779f81 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:22.715923 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0278a-998b-455f-ace7-6f54ff779f81 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:22.715923 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0278a-998b-455f-ace7-6f54ff779f81 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:22.717338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:22.722615 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c2c2c-ada9-41fb-aebd-121b3bdbbc47 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:22.723015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c2c2c-ada9-41fb-aebd-121b3bdbbc47 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:22.723193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c2c2c-ada9-41fb-aebd-121b3bdbbc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:22.762929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:22.788550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-872c2c2c-ada9-41fb-aebd-121b3bdbbc47 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:22.789768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-872c2c2c-ada9-41fb-aebd-121b3bdbbc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:22.884976 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:22.923900 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0278a-998b-455f-ace7-6f54ff779f81 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:22.925597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-01e0278a-998b-455f-ace7-6f54ff779f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:23.026851 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0278a-998b-455f-ace7-6f54ff779f81 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:23.057896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:23.100404 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b8fe6-2564-454e-80c7-668f64c53698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:23.101057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b8fe6-2564-454e-80c7-668f64c53698 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:23.101616 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b8fe6-2564-454e-80c7-668f64c53698 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:23.101616 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b8fe6-2564-454e-80c7-668f64c53698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:23.103268 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-460b8fe6-2564-454e-80c7-668f64c53698 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:23.209281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b8fe6-2564-454e-80c7-668f64c53698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:23.210707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b8fe6-2564-454e-80c7-668f64c53698 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:23.228866 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 716/1443] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:25.410154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06968f70-1233-4928-b8ec-9d883202d634 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:25.410725 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06968f70-1233-4928-b8ec-9d883202d634 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:25.410725 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06968f70-1233-4928-b8ec-9d883202d634 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:25.412046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:25.417355 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deda727-fca7-4bca-824a-bd5dc5f2d1d7 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:25.417781 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deda727-fca7-4bca-824a-bd5dc5f2d1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:25.417912 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deda727-fca7-4bca-824a-bd5dc5f2d1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:25.482046 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-1deda727-fca7-4bca-824a-bd5dc5f2d1d7 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:25.483273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-1deda727-fca7-4bca-824a-bd5dc5f2d1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:25.495752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:25.582802 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:25.620592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06968f70-1233-4928-b8ec-9d883202d634 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:25.622137 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-06968f70-1233-4928-b8ec-9d883202d634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:25.725570 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06968f70-1233-4928-b8ec-9d883202d634 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:25.737927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:25.799939 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10eb7c-0200-463c-834d-20552f80a5a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:25.800614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10eb7c-0200-463c-834d-20552f80a5a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:25.800902 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10eb7c-0200-463c-834d-20552f80a5a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:25.801180 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10eb7c-0200-463c-834d-20552f80a5a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:25.803463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bf10eb7c-0200-463c-834d-20552f80a5a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:25.913364 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10eb7c-0200-463c-834d-20552f80a5a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:25.914787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10eb7c-0200-463c-834d-20552f80a5a2 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:25.933699 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 729/1446] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:28.262385 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1c671-2d84-46b7-bf49-763a47a49c09 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:28.262941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1c671-2d84-46b7-bf49-763a47a49c09 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:28.262941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1c671-2d84-46b7-bf49-763a47a49c09 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:28.264344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:28.270287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e13c65-05cc-43f4-b234-5813f22362bc None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:28.270695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e13c65-05cc-43f4-b234-5813f22362bc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:28.270842 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e13c65-05cc-43f4-b234-5813f22362bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:28.317128 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:28.335816 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d4e13c65-05cc-43f4-b234-5813f22362bc None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:28.337074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d4e13c65-05cc-43f4-b234-5813f22362bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:28.432684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:28.471623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1c671-2d84-46b7-bf49-763a47a49c09 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:28.473194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-15d1c671-2d84-46b7-bf49-763a47a49c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:28.575111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1c671-2d84-46b7-bf49-763a47a49c09 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:28.614354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:28.648793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd456825-9fdf-4a33-96f4-c8d3d15a0266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:28.649438 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd456825-9fdf-4a33-96f4-c8d3d15a0266 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:28.649752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd456825-9fdf-4a33-96f4-c8d3d15a0266 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:28.650027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd456825-9fdf-4a33-96f4-c8d3d15a0266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:28.651682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-cd456825-9fdf-4a33-96f4-c8d3d15a0266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:28.758891 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd456825-9fdf-4a33-96f4-c8d3d15a0266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:28.760341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd456825-9fdf-4a33-96f4-c8d3d15a0266 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:28.777874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 719/1449] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:30.910562 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cc13f-8bef-4e48-8ccb-9d75f8f7c792 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:30.911100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cc13f-8bef-4e48-8ccb-9d75f8f7c792 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:30.911100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cc13f-8bef-4e48-8ccb-9d75f8f7c792 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:30.912550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:30.918059 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29eadfd-969d-47bc-ad7a-57e7f1e0f1a3 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:30.918503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29eadfd-969d-47bc-ad7a-57e7f1e0f1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:30.918616 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29eadfd-969d-47bc-ad7a-57e7f1e0f1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:30.981596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-b29eadfd-969d-47bc-ad7a-57e7f1e0f1a3 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:30.982806 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-b29eadfd-969d-47bc-ad7a-57e7f1e0f1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:31.003072 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:31.079484 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:31.118232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cc13f-8bef-4e48-8ccb-9d75f8f7c792 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:31.119883 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fe8cc13f-8bef-4e48-8ccb-9d75f8f7c792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:31.221471 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cc13f-8bef-4e48-8ccb-9d75f8f7c792 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:31.240981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:31.293285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efb437-21e4-4fab-83d9-b90a90a7d93d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:31.293950 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efb437-21e4-4fab-83d9-b90a90a7d93d service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:31.294231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efb437-21e4-4fab-83d9-b90a90a7d93d service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:31.294500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efb437-21e4-4fab-83d9-b90a90a7d93d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:31.296164 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d4efb437-21e4-4fab-83d9-b90a90a7d93d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:31.405174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efb437-21e4-4fab-83d9-b90a90a7d93d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:31.407187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efb437-21e4-4fab-83d9-b90a90a7d93d service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:31.427943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 732/1452] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:17:33.516279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597176e8-2dd4-47bd-aac5-7b4ccb8bdea9 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:33.516827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597176e8-2dd4-47bd-aac5-7b4ccb8bdea9 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:33.516827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597176e8-2dd4-47bd-aac5-7b4ccb8bdea9 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:33.518254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:33.523433 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51727e14-0230-40b1-bd7d-726518b1e363 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:33.523847 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51727e14-0230-40b1-bd7d-726518b1e363 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:33.524003 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51727e14-0230-40b1-bd7d-726518b1e363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:33.586509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:33.589431 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-51727e14-0230-40b1-bd7d-726518b1e363 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:33.590663 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-51727e14-0230-40b1-bd7d-726518b1e363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:33.686704 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:33.724820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597176e8-2dd4-47bd-aac5-7b4ccb8bdea9 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:33.726389 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-597176e8-2dd4-47bd-aac5-7b4ccb8bdea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:33.827897 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597176e8-2dd4-47bd-aac5-7b4ccb8bdea9 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:33.876627 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:33.902071 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9554f-52a2-49ff-a43d-72b350fc8f41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:33.902720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9554f-52a2-49ff-a43d-72b350fc8f41 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:33.903004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9554f-52a2-49ff-a43d-72b350fc8f41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:33.903276 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9554f-52a2-49ff-a43d-72b350fc8f41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:33.904963 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-bee9554f-52a2-49ff-a43d-72b350fc8f41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:34.011397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9554f-52a2-49ff-a43d-72b350fc8f41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:34.012858 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9554f-52a2-49ff-a43d-72b350fc8f41 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:34.030746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 722/1455] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:36.096885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca76e7b-34ef-4975-a499-1245aded8421 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:36.097478 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca76e7b-34ef-4975-a499-1245aded8421 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:36.097478 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca76e7b-34ef-4975-a499-1245aded8421 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:36.098836 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:36.104030 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2363ab-c5a7-4e72-8533-66e801a665ab None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:36.104472 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2363ab-c5a7-4e72-8533-66e801a665ab None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:36.104617 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2363ab-c5a7-4e72-8533-66e801a665ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:36.168173 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-ca2363ab-c5a7-4e72-8533-66e801a665ab None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:36.169415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ca2363ab-c5a7-4e72-8533-66e801a665ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:36.198936 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:36.266090 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:36.304083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca76e7b-34ef-4975-a499-1245aded8421 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:36.305647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1ca76e7b-34ef-4975-a499-1245aded8421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:36.408678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca76e7b-34ef-4975-a499-1245aded8421 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:36.436681 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:36.481421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755fa40-0524-45ff-9c5c-efc3d86eccc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:36.482062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755fa40-0524-45ff-9c5c-efc3d86eccc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:36.482340 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755fa40-0524-45ff-9c5c-efc3d86eccc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:36.482610 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755fa40-0524-45ff-9c5c-efc3d86eccc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:36.484247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a755fa40-0524-45ff-9c5c-efc3d86eccc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:36.591804 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755fa40-0524-45ff-9c5c-efc3d86eccc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:36.593290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755fa40-0524-45ff-9c5c-efc3d86eccc8 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:36.612130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 735/1458] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:38.761363 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d81719-accf-4283-a1a7-13ebd1524206 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:38.761942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d81719-accf-4283-a1a7-13ebd1524206 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:38.761942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d81719-accf-4283-a1a7-13ebd1524206 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:38.763190 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:38.769059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f672b7-d7d9-4056-9489-053e3d62f7ab None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:38.769468 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f672b7-d7d9-4056-9489-053e3d62f7ab None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:38.769612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f672b7-d7d9-4056-9489-053e3d62f7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:38.833473 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-67f672b7-d7d9-4056-9489-053e3d62f7ab None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:38.834678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-67f672b7-d7d9-4056-9489-053e3d62f7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:38.842868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:38.932299 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:38.970543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d81719-accf-4283-a1a7-13ebd1524206 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:38.972070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-82d81719-accf-4283-a1a7-13ebd1524206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:39.072120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d81719-accf-4283-a1a7-13ebd1524206 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:39.126244 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:39.145017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da404db-b569-4178-8376-882c85739ca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:39.145668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da404db-b569-4178-8376-882c85739ca3 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:39.145952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da404db-b569-4178-8376-882c85739ca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:39.146228 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da404db-b569-4178-8376-882c85739ca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:39.147890 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-8da404db-b569-4178-8376-882c85739ca3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:39.259944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da404db-b569-4178-8376-882c85739ca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:39.261413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da404db-b569-4178-8376-882c85739ca3 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:39.279083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 725/1461] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:41.346628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89309e43-240d-4536-a84c-30995887b3e3 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:41.347255 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89309e43-240d-4536-a84c-30995887b3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:41.347255 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89309e43-240d-4536-a84c-30995887b3e3 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:41.348662 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:41.354495 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae2588-6abe-4c62-96c9-4bab6e65cc17 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:41.354943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae2588-6abe-4c62-96c9-4bab6e65cc17 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:41.355058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae2588-6abe-4c62-96c9-4bab6e65cc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:41.419552 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-3dae2588-6abe-4c62-96c9-4bab6e65cc17 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:41.420713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-3dae2588-6abe-4c62-96c9-4bab6e65cc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:41.458125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:41.516934 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:41.554551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89309e43-240d-4536-a84c-30995887b3e3 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:41.556111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-89309e43-240d-4536-a84c-30995887b3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:41.663788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89309e43-240d-4536-a84c-30995887b3e3 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:41.704441 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:41.740036 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23767359-9514-48ab-89bd-3a704470b429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:41.740703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23767359-9514-48ab-89bd-3a704470b429 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:41.740984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23767359-9514-48ab-89bd-3a704470b429 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:41.741301 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23767359-9514-48ab-89bd-3a704470b429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:41.742968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-23767359-9514-48ab-89bd-3a704470b429 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:41.849483 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23767359-9514-48ab-89bd-3a704470b429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:41.851000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23767359-9514-48ab-89bd-3a704470b429 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:41.868925 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 738/1464] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:43.941790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f8bf5-18ce-46cf-b355-2ef76f3a9a07 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:43.942366 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f8bf5-18ce-46cf-b355-2ef76f3a9a07 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:43.942366 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f8bf5-18ce-46cf-b355-2ef76f3a9a07 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:43.943713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:43.949093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf797e-e966-4403-99ba-faa557a6c90e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:43.949937 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf797e-e966-4403-99ba-faa557a6c90e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:43.950243 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf797e-e966-4403-99ba-faa557a6c90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:44.015644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4edf797e-e966-4403-99ba-faa557a6c90e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:44.016890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4edf797e-e966-4403-99ba-faa557a6c90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:44.034535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:44.118170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:17:44.154518 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f8bf5-18ce-46cf-b355-2ef76f3a9a07 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:44.156070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-7b5f8bf5-18ce-46cf-b355-2ef76f3a9a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:44.263391 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f8bf5-18ce-46cf-b355-2ef76f3a9a07 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:44.331506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:44.339055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fe7c8-827f-4093-9627-afcc5aec8cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:44.339735 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fe7c8-827f-4093-9627-afcc5aec8cff service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:44.340016 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fe7c8-827f-4093-9627-afcc5aec8cff service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:44.340302 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fe7c8-827f-4093-9627-afcc5aec8cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:44.342018 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-7f3fe7c8-827f-4093-9627-afcc5aec8cff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:44.450806 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3fe7c8-827f-4093-9627-afcc5aec8cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:44.452153 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3fe7c8-827f-4093-9627-afcc5aec8cff service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:44.471330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 728/1467] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:46.523536 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5138d6-c0a0-47f7-b47e-81467d077557 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:46.524111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5138d6-c0a0-47f7-b47e-81467d077557 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:46.524111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5138d6-c0a0-47f7-b47e-81467d077557 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:46.525512 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:46.531168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9993826-f64e-405b-aa1c-04af7b3820ab None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:46.532351 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9993826-f64e-405b-aa1c-04af7b3820ab None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:46.532351 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9993826-f64e-405b-aa1c-04af7b3820ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:46.617524 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-e9993826-f64e-405b-aa1c-04af7b3820ab None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:46.619318 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e9993826-f64e-405b-aa1c-04af7b3820ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:46.671086 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:46.722602 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:46.762428 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5138d6-c0a0-47f7-b47e-81467d077557 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:46.764007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6c5138d6-c0a0-47f7-b47e-81467d077557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:46.875319 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5138d6-c0a0-47f7-b47e-81467d077557 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:46.924123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:46.950802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483ac08-2346-4283-9a6c-33cb20d7f06a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:46.951461 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483ac08-2346-4283-9a6c-33cb20d7f06a service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:46.951746 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483ac08-2346-4283-9a6c-33cb20d7f06a service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:46.952022 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483ac08-2346-4283-9a6c-33cb20d7f06a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:46.954291 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4483ac08-2346-4283-9a6c-33cb20d7f06a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:47.074738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483ac08-2346-4283-9a6c-33cb20d7f06a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:47.076195 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483ac08-2346-4283-9a6c-33cb20d7f06a service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:47.096936 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 741/1470] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:56.720775 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=22 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:57.093749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=66 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:06.807060 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32693e2a-40eb-49d4-9d30-7b8e860d400d None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:06.807638 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32693e2a-40eb-49d4-9d30-7b8e860d400d None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:06.807638 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32693e2a-40eb-49d4-9d30-7b8e860d400d None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:06.809280 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 730/1471] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:18:06.815052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491b66f-dd1f-46b4-a905-2c5e2f75f315 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:06.815511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491b66f-dd1f-46b4-a905-2c5e2f75f315 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:06.815617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491b66f-dd1f-46b4-a905-2c5e2f75f315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:06.881044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8491b66f-dd1f-46b4-a905-2c5e2f75f315 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:18:06.882289 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8491b66f-dd1f-46b4-a905-2c5e2f75f315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:06.979592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 742/1472] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:18:07.073097 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4d9b6-e0ca-41e0-b092-f74b82b43429 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:07.073384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4d9b6-e0ca-41e0-b092-f74b82b43429 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:07.073665 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4d9b6-e0ca-41e0-b092-f74b82b43429 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:07.075040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 731/1473] 10.222.0.43 () {54 vars in 966 bytes} [Wed Jul 20 19:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:18:07.080522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a0d02-f02d-4769-83c1-9550534f3023 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:07.080815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a0d02-f02d-4769-83c1-9550534f3023 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:07.081095 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a0d02-f02d-4769-83c1-9550534f3023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:07.159114 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-497a0d02-f02d-4769-83c1-9550534f3023 None None] MFA Rules not processed for user `d3e28557896e41c893f0de2590bc96d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:18:07.160362 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-497a0d02-f02d-4769-83c1-9550534f3023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:07.182345 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:07.239819 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 743/1474] 10.222.0.43 () {58 vars in 1063 bytes} [Wed Jul 20 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:18:07.247285 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4d9b6-e0ca-41e0-b092-f74b82b43429 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:07.248897 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-1bc4d9b6-e0ca-41e0-b092-f74b82b43429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:07.359061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4d9b6-e0ca-41e0-b092-f74b82b43429 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:07.434062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab90a5-9c57-47de-ac4b-5eb0e8af9273 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3e28557896e41c893f0de2590bc96d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:07.434717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab90a5-9c57-47de-ac4b-5eb0e8af9273 service glance] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:07.435005 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab90a5-9c57-47de-ac4b-5eb0e8af9273 service glance] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:07.435282 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab90a5-9c57-47de-ac4b-5eb0e8af9273 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:07.437006 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e6ab90a5-9c57-47de-ac4b-5eb0e8af9273 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:07.500919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:07.557313 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab90a5-9c57-47de-ac4b-5eb0e8af9273 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:07.559013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab90a5-9c57-47de-ac4b-5eb0e8af9273 service glance] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:07.579109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 732/1475] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:18:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:18:09.724935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b1778-b463-44d5-80fa-dd04cf931cef None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:09.725550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b1778-b463-44d5-80fa-dd04cf931cef None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:09.725550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b1778-b463-44d5-80fa-dd04cf931cef None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:09.726882 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 744/1476] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:18:09.732243 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77f16e-1ca6-484a-a027-15112a2aca18 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:09.732808 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77f16e-1ca6-484a-a027-15112a2aca18 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:09.732808 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77f16e-1ca6-484a-a027-15112a2aca18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:09.925958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-2e77f16e-1ca6-484a-a027-15112a2aca18 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:18:09.927176 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-2e77f16e-1ca6-484a-a027-15112a2aca18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:10.027977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 733/1477] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:18:10.053600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b1778-b463-44d5-80fa-dd04cf931cef None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:10.055297 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ea3b1778-b463-44d5-80fa-dd04cf931cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:10.186470 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b1778-b463-44d5-80fa-dd04cf931cef None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:10.237466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:10.294536 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531dbb3-9c52-4f07-9053-cc80dc4308cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:10.295181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531dbb3-9c52-4f07-9053-cc80dc4308cd admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:10.295472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531dbb3-9c52-4f07-9053-cc80dc4308cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:10.295769 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531dbb3-9c52-4f07-9053-cc80dc4308cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:10.296665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531dbb3-9c52-4f07-9053-cc80dc4308cd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:10.298658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: WARNING py.warnings [None req-5531dbb3-9c52-4f07-9053-cc80dc4308cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:18:10.298658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: warnings.warn(msg) Jul 20 19:18:10.298658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: Jul 20 19:18:10.299902 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531dbb3-9c52-4f07-9053-cc80dc4308cd admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:10.319688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 745/1478] 10.222.0.43 () {56 vars in 1108 bytes} [Wed Jul 20 19:18:10 2022] GET /identity/v3/projects => generated 2553 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:18:12.398656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15db0fe-a853-4302-92a3-47e469bff87f None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:12.399211 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15db0fe-a853-4302-92a3-47e469bff87f None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:12.399211 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15db0fe-a853-4302-92a3-47e469bff87f None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:12.400611 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 734/1479] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:18:12.405997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616f828-262b-45d8-b78e-ffa5ef133d68 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:12.406460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616f828-262b-45d8-b78e-ffa5ef133d68 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:12.406601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616f828-262b-45d8-b78e-ffa5ef133d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:12.472269 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b616f828-262b-45d8-b78e-ffa5ef133d68 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:18:12.473511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b616f828-262b-45d8-b78e-ffa5ef133d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:12.569040 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 746/1480] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:18:12.613720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15db0fe-a853-4302-92a3-47e469bff87f None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:12.615275 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f15db0fe-a853-4302-92a3-47e469bff87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:12.625113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:12.762113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15db0fe-a853-4302-92a3-47e469bff87f None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:12.841546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:12.874564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d712bc-5d7c-480d-b37f-7a9fd3c6687c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:12.875164 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d712bc-5d7c-480d-b37f-7a9fd3c6687c service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:12.875446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d712bc-5d7c-480d-b37f-7a9fd3c6687c service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:12.875720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d712bc-5d7c-480d-b37f-7a9fd3c6687c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:12.877415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-46d712bc-5d7c-480d-b37f-7a9fd3c6687c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:12.966861 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d712bc-5d7c-480d-b37f-7a9fd3c6687c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:12.968474 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d712bc-5d7c-480d-b37f-7a9fd3c6687c service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:12.987563 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 735/1481] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:18:15.223527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dab0f-cb19-4db1-9074-24c9a87fe16b None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:15.224034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dab0f-cb19-4db1-9074-24c9a87fe16b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:15.224141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dab0f-cb19-4db1-9074-24c9a87fe16b None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:15.225561 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 747/1482] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:18:15.231212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ce652-49c2-48fd-8796-0a7f2aadd6fc None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:15.231634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ce652-49c2-48fd-8796-0a7f2aadd6fc None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:15.231743 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ce652-49c2-48fd-8796-0a7f2aadd6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:15.294336 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-478ce652-49c2-48fd-8796-0a7f2aadd6fc None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:18:15.295523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-478ce652-49c2-48fd-8796-0a7f2aadd6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:15.340840 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:15.391345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 736/1483] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:18:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:18:15.428547 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dab0f-cb19-4db1-9074-24c9a87fe16b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:15.430123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5f4dab0f-cb19-4db1-9074-24c9a87fe16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:15.455844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:15.536528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dab0f-cb19-4db1-9074-24c9a87fe16b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:15.611689 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524b612-c7a9-4a87-8a3a-54e93e0fbaaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:15.612357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524b612-c7a9-4a87-8a3a-54e93e0fbaaa service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:15.612639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524b612-c7a9-4a87-8a3a-54e93e0fbaaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:15.612908 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524b612-c7a9-4a87-8a3a-54e93e0fbaaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:15.614567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e524b612-c7a9-4a87-8a3a-54e93e0fbaaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:15.693675 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:15.734630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e524b612-c7a9-4a87-8a3a-54e93e0fbaaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:15.736537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e524b612-c7a9-4a87-8a3a-54e93e0fbaaa service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:15.757589 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 748/1484] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:18:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:18:17.855355 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e2b15-a853-4909-97f0-ddf78a67ee6b None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:17.855864 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e2b15-a853-4909-97f0-ddf78a67ee6b None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:17.855864 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e2b15-a853-4909-97f0-ddf78a67ee6b None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:17.857206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 737/1485] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:18:17.863031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47be6bd-f1aa-44df-920d-3ccc6d5a1e2e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:17.863434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47be6bd-f1aa-44df-920d-3ccc6d5a1e2e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:17.863592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47be6bd-f1aa-44df-920d-3ccc6d5a1e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:17.928886 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a47be6bd-f1aa-44df-920d-3ccc6d5a1e2e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:18:17.930113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a47be6bd-f1aa-44df-920d-3ccc6d5a1e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:18.027393 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 749/1486] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:18:18.067042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e2b15-a853-4909-97f0-ddf78a67ee6b None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:18.068706 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d00e2b15-a853-4909-97f0-ddf78a67ee6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:18.170285 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e2b15-a853-4909-97f0-ddf78a67ee6b None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:18.245980 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:18.247890 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6836e-f5a8-4b64-81c1-a33f9a50f086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:18.248614 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6836e-f5a8-4b64-81c1-a33f9a50f086 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:18.249154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6836e-f5a8-4b64-81c1-a33f9a50f086 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:18.249695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6836e-f5a8-4b64-81c1-a33f9a50f086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:18.251592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-49c6836e-f5a8-4b64-81c1-a33f9a50f086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:18.360283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6836e-f5a8-4b64-81c1-a33f9a50f086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:18.361727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6836e-f5a8-4b64-81c1-a33f9a50f086 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:18.380487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 738/1487] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:18:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:18:20.574256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2958fd-81c6-45ec-91d3-f12db23cba79 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:20.574744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2958fd-81c6-45ec-91d3-f12db23cba79 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:20.574843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2958fd-81c6-45ec-91d3-f12db23cba79 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:20.576222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 750/1488] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:18:20.581806 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65624bd-2180-4672-9f39-4c852b418805 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:20.582244 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65624bd-2180-4672-9f39-4c852b418805 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:20.582351 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65624bd-2180-4672-9f39-4c852b418805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:20.647189 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-e65624bd-2180-4672-9f39-4c852b418805 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:18:20.648351 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e65624bd-2180-4672-9f39-4c852b418805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:20.703791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:20.747431 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 739/1489] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:18:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:18:20.787463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2958fd-81c6-45ec-91d3-f12db23cba79 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:20.789086 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cb2958fd-81c6-45ec-91d3-f12db23cba79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:20.827288 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:20.894466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2958fd-81c6-45ec-91d3-f12db23cba79 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:20.967015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f649c-bfaa-4ca8-8dba-84094257522d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:20.967577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f649c-bfaa-4ca8-8dba-84094257522d service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:20.967858 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f649c-bfaa-4ca8-8dba-84094257522d service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:20.968134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f649c-bfaa-4ca8-8dba-84094257522d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:20.970324 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3d5f649c-bfaa-4ca8-8dba-84094257522d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:21.048101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:21.077595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f649c-bfaa-4ca8-8dba-84094257522d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:21.079592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f649c-bfaa-4ca8-8dba-84094257522d service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:21.097558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 751/1490] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:18:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:18:23.355424 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b425e-04be-4130-9480-81767975be9f None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:23.355829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b425e-04be-4130-9480-81767975be9f None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:23.355976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b425e-04be-4130-9480-81767975be9f None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:23.357414 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 740/1491] 10.222.0.43 () {54 vars in 926 bytes} [Wed Jul 20 19:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:18:23.363120 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b7831-3ff6-4af9-8360-fe5c5edbbc13 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:23.363540 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b7831-3ff6-4af9-8360-fe5c5edbbc13 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:23.363682 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b7831-3ff6-4af9-8360-fe5c5edbbc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:23.430614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f77b7831-3ff6-4af9-8360-fe5c5edbbc13 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:18:23.431827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f77b7831-3ff6-4af9-8360-fe5c5edbbc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:23.528734 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 752/1492] 10.222.0.43 () {58 vars in 1023 bytes} [Wed Jul 20 19:18:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:18:23.569250 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b425e-04be-4130-9480-81767975be9f None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:23.570755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4b9b425e-04be-4130-9480-81767975be9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:23.672273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b425e-04be-4130-9480-81767975be9f None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:23.744863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91b186-1563-4db7-8f33-daf8bf319e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:23.745555 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91b186-1563-4db7-8f33-daf8bf319e7b service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:23.745834 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91b186-1563-4db7-8f33-daf8bf319e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:23.746105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91b186-1563-4db7-8f33-daf8bf319e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:23.747755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0e91b186-1563-4db7-8f33-daf8bf319e7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:23.756813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:23.856651 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91b186-1563-4db7-8f33-daf8bf319e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:23.858089 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91b186-1563-4db7-8f33-daf8bf319e7b service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:23.876841 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 741/1493] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:18:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:18:33.526894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=82 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:33.874470 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=50 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:49.826853 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552169f-1a40-4daa-a3a5-96c081ca4b58 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:49.828005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552169f-1a40-4daa-a3a5-96c081ca4b58 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:49.828005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552169f-1a40-4daa-a3a5-96c081ca4b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:49.900413 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5552169f-1a40-4daa-a3a5-96c081ca4b58 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:49.901611 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5552169f-1a40-4daa-a3a5-96c081ca4b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:49.998881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 753/1494] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:50.007666 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e89b76-ce34-4061-8fa4-9300a99f09c5 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:50.008182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e89b76-ce34-4061-8fa4-9300a99f09c5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:50.008319 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e89b76-ce34-4061-8fa4-9300a99f09c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:50.087191 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-f6e89b76-ce34-4061-8fa4-9300a99f09c5 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:50.088425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f6e89b76-ce34-4061-8fa4-9300a99f09c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:50.182932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 742/1495] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:50.188461 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552169f-1a40-4daa-a3a5-96c081ca4b58 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:50.189672 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5552169f-1a40-4daa-a3a5-96c081ca4b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:50.201506 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552169f-1a40-4daa-a3a5-96c081ca4b58 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:50.262892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:50.305423 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412330fa-63f1-4167-ae33-53bc189df506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:50.306080 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412330fa-63f1-4167-ae33-53bc189df506 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:50.306358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412330fa-63f1-4167-ae33-53bc189df506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:50.306627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412330fa-63f1-4167-ae33-53bc189df506 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:50.307347 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412330fa-63f1-4167-ae33-53bc189df506 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:19:50.307652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412330fa-63f1-4167-ae33-53bc189df506 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:50.308702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: WARNING py.warnings [None req-412330fa-63f1-4167-ae33-53bc189df506 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:19:50.308702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: warnings.warn(msg) Jul 20 19:19:50.308702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: Jul 20 19:19:50.309830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412330fa-63f1-4167-ae33-53bc189df506 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:50.324558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 754/1496] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:19:50.333453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e89b76-ce34-4061-8fa4-9300a99f09c5 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:50.343508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e89b76-ce34-4061-8fa4-9300a99f09c5 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:50.404794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:50.439249 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3e48a-5320-49f1-97df-b16d2f351e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:50.439914 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3e48a-5320-49f1-97df-b16d2f351e2e admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:50.440196 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3e48a-5320-49f1-97df-b16d2f351e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:50.440499 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3e48a-5320-49f1-97df-b16d2f351e2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:50.441183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3e48a-5320-49f1-97df-b16d2f351e2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:50.442909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: WARNING py.warnings [None req-48b3e48a-5320-49f1-97df-b16d2f351e2e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:19:50.442909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: warnings.warn(msg) Jul 20 19:19:50.442909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: Jul 20 19:19:50.444103 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3e48a-5320-49f1-97df-b16d2f351e2e admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:50.498310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 743/1497] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:19:50 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:19:50.503145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412330fa-63f1-4167-ae33-53bc189df506 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:50.504833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-412330fa-63f1-4167-ae33-53bc189df506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:50.541066 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412330fa-63f1-4167-ae33-53bc189df506 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:50.637489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85171321-302f-4ad6-bd82-be0b310f0d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:50.638078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85171321-302f-4ad6-bd82-be0b310f0d9f admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:50.638362 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85171321-302f-4ad6-bd82-be0b310f0d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:50.638639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85171321-302f-4ad6-bd82-be0b310f0d9f admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:50.639344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85171321-302f-4ad6-bd82-be0b310f0d9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:50.640433 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: WARNING py.warnings [None req-85171321-302f-4ad6-bd82-be0b310f0d9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:19:50.640433 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: warnings.warn(msg) Jul 20 19:19:50.640433 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: Jul 20 19:19:50.641632 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85171321-302f-4ad6-bd82-be0b310f0d9f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:50.683089 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 755/1498] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:19:50 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:19:50.687559 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3e48a-5320-49f1-97df-b16d2f351e2e admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:50.689211 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-48b3e48a-5320-49f1-97df-b16d2f351e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:50.753080 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3e48a-5320-49f1-97df-b16d2f351e2e admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:50.837970 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:50.849557 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233fb30-ae23-44d5-8404-2905a7cd45ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:50.850229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233fb30-ae23-44d5-8404-2905a7cd45ee admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:50.850512 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233fb30-ae23-44d5-8404-2905a7cd45ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:50.850785 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233fb30-ae23-44d5-8404-2905a7cd45ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:50.851515 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233fb30-ae23-44d5-8404-2905a7cd45ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:50.852546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: WARNING py.warnings [None req-6233fb30-ae23-44d5-8404-2905a7cd45ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:19:50.852546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: warnings.warn(msg) Jul 20 19:19:50.852546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: Jul 20 19:19:50.853441 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233fb30-ae23-44d5-8404-2905a7cd45ee admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:50.865964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 744/1499] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:19:50.870573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85171321-302f-4ad6-bd82-be0b310f0d9f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:50.872149 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-85171321-302f-4ad6-bd82-be0b310f0d9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:50.886551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85171321-302f-4ad6-bd82-be0b310f0d9f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:50.913406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:50.985943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6bc65-bbd0-48f8-b20d-863bf303a5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:50.986611 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6bc65-bbd0-48f8-b20d-863bf303a5fb admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:50.986892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6bc65-bbd0-48f8-b20d-863bf303a5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:50.987163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6bc65-bbd0-48f8-b20d-863bf303a5fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:50.987892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6bc65-bbd0-48f8-b20d-863bf303a5fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:50.988947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: WARNING py.warnings [None req-d9c6bc65-bbd0-48f8-b20d-863bf303a5fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:19:50.988947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: warnings.warn(msg) Jul 20 19:19:50.988947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: Jul 20 19:19:50.989827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6bc65-bbd0-48f8-b20d-863bf303a5fb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:51.002973 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 756/1500] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:19:51.007435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233fb30-ae23-44d5-8404-2905a7cd45ee admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:51.009040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6233fb30-ae23-44d5-8404-2905a7cd45ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:51.043266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233fb30-ae23-44d5-8404-2905a7cd45ee admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:51.140032 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e28c4-d28a-4bb8-9eeb-01587e1a73ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:51.140941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e28c4-d28a-4bb8-9eeb-01587e1a73ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:51.141219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e28c4-d28a-4bb8-9eeb-01587e1a73ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:51.141488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e28c4-d28a-4bb8-9eeb-01587e1a73ed admin admin] PATH_INFO: `/v3/projects/bb2e42db20754da4b0fc4007e03f15e2/users/859d0f5f33ed4839914a631a975dbd41/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:51.163033 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e28c4-d28a-4bb8-9eeb-01587e1a73ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb2e42db20754da4b0fc4007e03f15e2, user_id=859d0f5f33ed4839914a631a975dbd41, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:51.164104 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: WARNING py.warnings [None req-2e5e28c4-d28a-4bb8-9eeb-01587e1a73ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:19:51.164104 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: warnings.warn(msg) Jul 20 19:19:51.164104 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: Jul 20 19:19:51.165999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e28c4-d28a-4bb8-9eeb-01587e1a73ed admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:51.195475 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 745/1501] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:19:51 2022] PUT /identity/v3/projects/bb2e42db20754da4b0fc4007e03f15e2/users/859d0f5f33ed4839914a631a975dbd41/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:19:51.207567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb612a-6495-455c-b6ea-039e08ce3591 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:51.207858 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb612a-6495-455c-b6ea-039e08ce3591 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:51.208135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb612a-6495-455c-b6ea-039e08ce3591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:51.263997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-78cb612a-6495-455c-b6ea-039e08ce3591 None None] MFA Rules not processed for user `859d0f5f33ed4839914a631a975dbd41`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:51.265250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-78cb612a-6495-455c-b6ea-039e08ce3591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:51.344165 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 757/1502] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:19:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:51.357681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3280ce7-89f0-41c0-87de-ceac619bb964 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:51.357992 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3280ce7-89f0-41c0-87de-ceac619bb964 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:51.358255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3280ce7-89f0-41c0-87de-ceac619bb964 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:51.360137 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 746/1503] 10.222.0.43 () {54 vars in 964 bytes} [Wed Jul 20 19:19:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:51.364997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537859e-5929-4d90-ab55-81b5c3e26e2d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:51.365286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537859e-5929-4d90-ab55-81b5c3e26e2d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:51.365562 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537859e-5929-4d90-ab55-81b5c3e26e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:51.515313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:51.562399 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f537859e-5929-4d90-ab55-81b5c3e26e2d None None] MFA Rules not processed for user `69aef294bb4943c3822ad7d0a3d91fd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:51.563592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f537859e-5929-4d90-ab55-81b5c3e26e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:51.638288 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 758/1504] 10.222.0.43 () {58 vars in 1061 bytes} [Wed Jul 20 19:19:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:51.646032 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3280ce7-89f0-41c0-87de-ceac619bb964 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:51.647547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f3280ce7-89f0-41c0-87de-ceac619bb964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:51.703151 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:51.759867 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3280ce7-89f0-41c0-87de-ceac619bb964 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:51.832336 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e016f-3a7a-42b4-86a3-feee5d23d3f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:51.832988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e016f-3a7a-42b4-86a3-feee5d23d3f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:51.833272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e016f-3a7a-42b4-86a3-feee5d23d3f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:51.833554 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e016f-3a7a-42b4-86a3-feee5d23d3f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:51.835211 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-fc8e016f-3a7a-42b4-86a3-feee5d23d3f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:51.933644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e016f-3a7a-42b4-86a3-feee5d23d3f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:51.935342 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e016f-3a7a-42b4-86a3-feee5d23d3f7 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:51.938943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:51.954438 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 747/1505] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:19:52.001941 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0b57b-29cc-444d-936f-5b9d30103e21 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:52.002394 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0b57b-29cc-444d-936f-5b9d30103e21 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:52.002501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0b57b-29cc-444d-936f-5b9d30103e21 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:52.003794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 759/1506] 10.222.0.43 () {52 vars in 829 bytes} [Wed Jul 20 19:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:52.007794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e016f-3a7a-42b4-86a3-feee5d23d3f7 service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:52.009059 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-fc8e016f-3a7a-42b4-86a3-feee5d23d3f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:52.111426 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e016f-3a7a-42b4-86a3-feee5d23d3f7 service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:52.205940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842994d-ce9f-4be5-a972-149460d4df08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:52.206723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842994d-ce9f-4be5-a972-149460d4df08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:52.206998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842994d-ce9f-4be5-a972-149460d4df08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:52.207265 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842994d-ce9f-4be5-a972-149460d4df08 admin admin] PATH_INFO: `/v3/users/859d0f5f33ed4839914a631a975dbd41` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:52.220190 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842994d-ce9f-4be5-a972-149460d4df08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=859d0f5f33ed4839914a631a975dbd41)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:52.221235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: WARNING py.warnings [None req-b842994d-ce9f-4be5-a972-149460d4df08 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:19:52.221235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: warnings.warn(msg) Jul 20 19:19:52.221235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: Jul 20 19:19:52.222438 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842994d-ce9f-4be5-a972-149460d4df08 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:52.232226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:52.289154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b842994d-ce9f-4be5-a972-149460d4df08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '859d0f5f33ed4839914a631a975dbd41'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:52.299286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b842994d-ce9f-4be5-a972-149460d4df08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '859d0f5f33ed4839914a631a975dbd41'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:52.319215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b842994d-ce9f-4be5-a972-149460d4df08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '859d0f5f33ed4839914a631a975dbd41'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:52.331266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b842994d-ce9f-4be5-a972-149460d4df08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '859d0f5f33ed4839914a631a975dbd41'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:52.331751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b842994d-ce9f-4be5-a972-149460d4df08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '859d0f5f33ed4839914a631a975dbd41'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:52.337699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 748/1507] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:19:52 2022] DELETE /identity/v3/users/859d0f5f33ed4839914a631a975dbd41 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:19:52.342184 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76630f53-273f-40d9-976b-5b2deedf33c6 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:52.342469 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76630f53-273f-40d9-976b-5b2deedf33c6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:52.343240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76630f53-273f-40d9-976b-5b2deedf33c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:52.400768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-76630f53-273f-40d9-976b-5b2deedf33c6 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:52.402012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-76630f53-273f-40d9-976b-5b2deedf33c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:52.480024 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:52.493992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 760/1508] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:52.503776 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842994d-ce9f-4be5-a972-149460d4df08 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:52.505048 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-b842994d-ce9f-4be5-a972-149460d4df08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:52.613369 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842994d-ce9f-4be5-a972-149460d4df08 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:52.675562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:52.696030 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d64c27-5a55-49e0-ba43-522695a3b53c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:52.696790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d64c27-5a55-49e0-ba43-522695a3b53c service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:52.697157 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d64c27-5a55-49e0-ba43-522695a3b53c service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:52.697433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d64c27-5a55-49e0-ba43-522695a3b53c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:52.699099 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-67d64c27-5a55-49e0-ba43-522695a3b53c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:52.807760 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d64c27-5a55-49e0-ba43-522695a3b53c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:52.809235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d64c27-5a55-49e0-ba43-522695a3b53c service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:52.830388 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 749/1509] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:19:53.194679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76630f53-273f-40d9-976b-5b2deedf33c6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:53.195924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-76630f53-273f-40d9-976b-5b2deedf33c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:53.208267 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76630f53-273f-40d9-976b-5b2deedf33c6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:53.314759 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9537de-034d-4450-a478-702cc0687514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:53.315619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9537de-034d-4450-a478-702cc0687514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:53.315898 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9537de-034d-4450-a478-702cc0687514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:53.316169 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9537de-034d-4450-a478-702cc0687514 admin admin] PATH_INFO: `/v3/projects/bb2e42db20754da4b0fc4007e03f15e2` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:53.326901 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9537de-034d-4450-a478-702cc0687514 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb2e42db20754da4b0fc4007e03f15e2)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:53.327885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: WARNING py.warnings [None req-ad9537de-034d-4450-a478-702cc0687514 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:19:53.327885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: warnings.warn(msg) Jul 20 19:19:53.327885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: Jul 20 19:19:53.329099 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9537de-034d-4450-a478-702cc0687514 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:53.388769 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ad9537de-034d-4450-a478-702cc0687514 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb2e42db20754da4b0fc4007e03f15e2', 'request_id': 'req-ad9537de-034d-4450-a478-702cc0687514'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:53.401885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ad9537de-034d-4450-a478-702cc0687514 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb2e42db20754da4b0fc4007e03f15e2', 'request_id': 'req-ad9537de-034d-4450-a478-702cc0687514'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:53.412509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ad9537de-034d-4450-a478-702cc0687514 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb2e42db20754da4b0fc4007e03f15e2', 'request_id': 'req-ad9537de-034d-4450-a478-702cc0687514'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:53.439043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ad9537de-034d-4450-a478-702cc0687514 admin admin] The token cache is being invalidate because project bb2e42db20754da4b0fc4007e03f15e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:19:53.439326 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ad9537de-034d-4450-a478-702cc0687514 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:53.439755 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ad9537de-034d-4450-a478-702cc0687514 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:53.441257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 761/1510] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:19:53 2022] DELETE /identity/v3/projects/bb2e42db20754da4b0fc4007e03f15e2 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:20:02.827463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=62 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:03.436178 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:39.543901 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015acf5-9069-41ad-850a-2d510b64f18f None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:39.544515 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015acf5-9069-41ad-850a-2d510b64f18f None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:39.544515 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015acf5-9069-41ad-850a-2d510b64f18f None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:39.545854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:39.550893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670b9eb-f729-4c50-b589-6497a801d309 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:39.551799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670b9eb-f729-4c50-b589-6497a801d309 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:39.551902 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670b9eb-f729-4c50-b589-6497a801d309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:39.632064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f670b9eb-f729-4c50-b589-6497a801d309 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:39.633298 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f670b9eb-f729-4c50-b589-6497a801d309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:39.733634 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:39.884486 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015acf5-9069-41ad-850a-2d510b64f18f None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:39.886561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f015acf5-9069-41ad-850a-2d510b64f18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:40.003945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015acf5-9069-41ad-850a-2d510b64f18f None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:40.009500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7f0f7-d197-43b1-b007-19a148a69f63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:40.010151 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7f0f7-d197-43b1-b007-19a148a69f63 service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:40.010440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7f0f7-d197-43b1-b007-19a148a69f63 service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:40.010720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7f0f7-d197-43b1-b007-19a148a69f63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:40.018824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7f0f7-d197-43b1-b007-19a148a69f63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:40.020980 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7f0f7-d197-43b1-b007-19a148a69f63 service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:40.030294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 751/1513] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:40.850437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b821f-42b6-4751-b1c7-e812b1e25df3 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:40.851023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b821f-42b6-4751-b1c7-e812b1e25df3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:40.851023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b821f-42b6-4751-b1c7-e812b1e25df3 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:40.852400 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:40.857305 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae08c3-b55e-4261-8ba0-8d101f50d440 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:40.857746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae08c3-b55e-4261-8ba0-8d101f50d440 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:40.857861 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae08c3-b55e-4261-8ba0-8d101f50d440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:40.880729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-28ae08c3-b55e-4261-8ba0-8d101f50d440 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:40.881839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-28ae08c3-b55e-4261-8ba0-8d101f50d440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:40.889791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:41.042082 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b821f-42b6-4751-b1c7-e812b1e25df3 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:41.057193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b821f-42b6-4751-b1c7-e812b1e25df3 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:41.062846 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000cfdce-6ca9-4648-8779-70d1b9b6eb20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:41.063482 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cfdce-6ca9-4648-8779-70d1b9b6eb20 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:41.063769 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cfdce-6ca9-4648-8779-70d1b9b6eb20 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:41.064052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cfdce-6ca9-4648-8779-70d1b9b6eb20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:41.072774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000cfdce-6ca9-4648-8779-70d1b9b6eb20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:41.074347 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000cfdce-6ca9-4648-8779-70d1b9b6eb20 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:41.083723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 764/1516] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:42.279922 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508819-a0bf-4ec7-9dd1-9591cea89d10 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:42.280493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508819-a0bf-4ec7-9dd1-9591cea89d10 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:42.280493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508819-a0bf-4ec7-9dd1-9591cea89d10 None None] PATH_INFO: `/v3` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:42.281809 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 753/1517] 10.222.0.89 () {54 vars in 925 bytes} [Wed Jul 20 19:21:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:21:42.300594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d4eee-0b6b-4185-9f95-f862948c0490 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:42.302767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d4eee-0b6b-4185-9f95-f862948c0490 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:42.302767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d4eee-0b6b-4185-9f95-f862948c0490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:42.359739 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d16d4eee-0b6b-4185-9f95-f862948c0490 None None] MFA Rules not processed for user `ae581d3dda354e07894cb5205d1b2c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:42.360976 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d16d4eee-0b6b-4185-9f95-f862948c0490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:42.377888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:42.432969 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 765/1518] 10.222.0.89 () {58 vars in 1015 bytes} [Wed Jul 20 19:21:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:21:46.451544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece33a19-dbf7-41f1-bcb8-73416afe8a2f None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:46.452106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece33a19-dbf7-41f1-bcb8-73416afe8a2f None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:46.452106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece33a19-dbf7-41f1-bcb8-73416afe8a2f None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:46.453489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 754/1519] 10.222.0.43 () {54 vars in 961 bytes} [Wed Jul 20 19:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:46.458956 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5984cfe-9585-42a9-9768-1ba35e7c8908 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:46.459418 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5984cfe-9585-42a9-9768-1ba35e7c8908 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:46.459522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5984cfe-9585-42a9-9768-1ba35e7c8908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:46.477197 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f5984cfe-9585-42a9-9768-1ba35e7c8908 None None] MFA Rules not processed for user `ae581d3dda354e07894cb5205d1b2c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:46.478384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f5984cfe-9585-42a9-9768-1ba35e7c8908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:46.486970 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 766/1520] 10.222.0.43 () {58 vars in 1058 bytes} [Wed Jul 20 19:21:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:46.494432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece33a19-dbf7-41f1-bcb8-73416afe8a2f None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:46.508182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece33a19-dbf7-41f1-bcb8-73416afe8a2f None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:46.513074 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5498a5-af81-441a-839c-4fd3e7b35e19 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae581d3dda354e07894cb5205d1b2c5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:46.513700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5498a5-af81-441a-839c-4fd3e7b35e19 service placement] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:46.513981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5498a5-af81-441a-839c-4fd3e7b35e19 service placement] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:46.514252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5498a5-af81-441a-839c-4fd3e7b35e19 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:46.523588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5498a5-af81-441a-839c-4fd3e7b35e19 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:46.525187 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5498a5-af81-441a-839c-4fd3e7b35e19 service placement] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:46.534857 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 755/1521] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:21:52.305453 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe4166-452d-430f-8a96-413744592fad None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:52.305964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe4166-452d-430f-8a96-413744592fad None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:52.306066 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe4166-452d-430f-8a96-413744592fad None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:52.307389 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:52.312644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5f52f-1606-4d2a-b13f-121f0806d5b5 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:52.313108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5f52f-1606-4d2a-b13f-121f0806d5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:52.313243 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5f52f-1606-4d2a-b13f-121f0806d5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:52.330756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-15e5f52f-1606-4d2a-b13f-121f0806d5b5 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:52.332986 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-15e5f52f-1606-4d2a-b13f-121f0806d5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:52.340947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:52.496542 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe4166-452d-430f-8a96-413744592fad None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:52.511798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe4166-452d-430f-8a96-413744592fad None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:52.517236 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd962af4-7a67-40ab-b960-4075b7e5e4eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:52.517870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd962af4-7a67-40ab-b960-4075b7e5e4eb service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:52.518154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd962af4-7a67-40ab-b960-4075b7e5e4eb service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:52.518431 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd962af4-7a67-40ab-b960-4075b7e5e4eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:52.527168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd962af4-7a67-40ab-b960-4075b7e5e4eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:52.528880 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd962af4-7a67-40ab-b960-4075b7e5e4eb service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:52.538378 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 768/1524] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:21:53.356567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c888481-1592-47a5-b004-cc7109c28f27 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:53.357013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c888481-1592-47a5-b004-cc7109c28f27 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:53.357111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c888481-1592-47a5-b004-cc7109c28f27 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:53.358527 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:53.364463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83600b0-c8a5-42a9-a876-6d55e9c07d43 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:53.364921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83600b0-c8a5-42a9-a876-6d55e9c07d43 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:53.365026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83600b0-c8a5-42a9-a876-6d55e9c07d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:53.383025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d83600b0-c8a5-42a9-a876-6d55e9c07d43 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:53.384242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d83600b0-c8a5-42a9-a876-6d55e9c07d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:53.393690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:53.547610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c888481-1592-47a5-b004-cc7109c28f27 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:53.562065 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c888481-1592-47a5-b004-cc7109c28f27 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:53.567715 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcf339-d73d-44d6-b6dd-bb7ddbd1e39f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:53.568394 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcf339-d73d-44d6-b6dd-bb7ddbd1e39f service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:53.568677 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcf339-d73d-44d6-b6dd-bb7ddbd1e39f service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:53.568946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcf339-d73d-44d6-b6dd-bb7ddbd1e39f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:53.578447 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcf339-d73d-44d6-b6dd-bb7ddbd1e39f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:53.580036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcf339-d73d-44d6-b6dd-bb7ddbd1e39f service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:53.591028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 758/1527] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:57.773336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b943a6-0c3f-433d-8909-c78407f12baf None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:57.773906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b943a6-0c3f-433d-8909-c78407f12baf None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:57.773906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b943a6-0c3f-433d-8909-c78407f12baf None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:57.775245 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:57.780418 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b85b4-5e29-4937-8966-5ffb7d0f6ea7 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:57.780872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b85b4-5e29-4937-8966-5ffb7d0f6ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:57.780976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b85b4-5e29-4937-8966-5ffb7d0f6ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:57.798328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-d57b85b4-5e29-4937-8966-5ffb7d0f6ea7 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:57.799513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d57b85b4-5e29-4937-8966-5ffb7d0f6ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:57.807503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:21:57.880956 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b943a6-0c3f-433d-8909-c78407f12baf None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:57.882683 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-10b943a6-0c3f-433d-8909-c78407f12baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:57.959952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b943a6-0c3f-433d-8909-c78407f12baf None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:57.963520 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70755fed-0b76-4c06-b81c-f570a8ede2f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:57.964156 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70755fed-0b76-4c06-b81c-f570a8ede2f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:57.964474 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70755fed-0b76-4c06-b81c-f570a8ede2f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:57.964753 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70755fed-0b76-4c06-b81c-f570a8ede2f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:57.973654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70755fed-0b76-4c06-b81c-f570a8ede2f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:57.975086 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70755fed-0b76-4c06-b81c-f570a8ede2f6 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:57.984401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 771/1530] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:59.987175 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb1767-59d3-4ac9-b923-590339a5f9b1 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:59.987733 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb1767-59d3-4ac9-b923-590339a5f9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:59.987733 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb1767-59d3-4ac9-b923-590339a5f9b1 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:59.989814 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:59.994905 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2611cd-0763-43e7-a0b0-083eb6dfb403 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:59.995353 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2611cd-0763-43e7-a0b0-083eb6dfb403 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:59.995460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2611cd-0763-43e7-a0b0-083eb6dfb403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:00.015773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-aa2611cd-0763-43e7-a0b0-083eb6dfb403 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:22:00.017007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-aa2611cd-0763-43e7-a0b0-083eb6dfb403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:00.024830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:22:00.061799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb1767-59d3-4ac9-b923-590339a5f9b1 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:22:00.072323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb1767-59d3-4ac9-b923-590339a5f9b1 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:22:00.075749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5f74f-cf0d-4f9d-a60d-8daeec573628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:22:00.076400 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5f74f-cf0d-4f9d-a60d-8daeec573628 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:00.076684 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5f74f-cf0d-4f9d-a60d-8daeec573628 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:00.076958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5f74f-cf0d-4f9d-a60d-8daeec573628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:00.085643 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5f74f-cf0d-4f9d-a60d-8daeec573628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:22:00.087036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5f74f-cf0d-4f9d-a60d-8daeec573628 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:22:00.096190 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 761/1533] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:22:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:22:04.161680 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23687b73-02f9-4170-ac13-7d82a0494d82 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:04.162258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23687b73-02f9-4170-ac13-7d82a0494d82 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:04.162258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23687b73-02f9-4170-ac13-7d82a0494d82 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:04.163633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:22:04.169033 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fe9a5-512c-4f8c-89d2-a3161991308c None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:04.169454 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fe9a5-512c-4f8c-89d2-a3161991308c None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:04.169594 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fe9a5-512c-4f8c-89d2-a3161991308c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:04.176518 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=81 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:04.189396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-218fe9a5-512c-4f8c-89d2-a3161991308c None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:22:04.190609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-218fe9a5-512c-4f8c-89d2-a3161991308c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:04.199098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:22:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:22:04.511564 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23687b73-02f9-4170-ac13-7d82a0494d82 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:22:04.513279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-23687b73-02f9-4170-ac13-7d82a0494d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:04.579375 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:04.593245 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23687b73-02f9-4170-ac13-7d82a0494d82 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:22:04.597467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc126f1-32cd-46de-b9fc-261e86a0852c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:22:04.598750 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc126f1-32cd-46de-b9fc-261e86a0852c service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:04.599027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc126f1-32cd-46de-b9fc-261e86a0852c service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:04.599298 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc126f1-32cd-46de-b9fc-261e86a0852c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:04.608412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc126f1-32cd-46de-b9fc-261e86a0852c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:22:04.609825 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc126f1-32cd-46de-b9fc-261e86a0852c service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:22:04.620127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 774/1536] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:22:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:22:06.673509 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee45a37-fb1b-4d8d-8aa8-8c8ab39f8f54 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:06.674014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee45a37-fb1b-4d8d-8aa8-8c8ab39f8f54 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:06.674014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee45a37-fb1b-4d8d-8aa8-8c8ab39f8f54 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:06.675349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:22:06.680688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9d79f-7441-4bed-8238-977978a53119 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:06.681132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9d79f-7441-4bed-8238-977978a53119 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:06.681242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9d79f-7441-4bed-8238-977978a53119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:06.699571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b9b9d79f-7441-4bed-8238-977978a53119 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:22:06.700792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b9b9d79f-7441-4bed-8238-977978a53119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:06.709440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:22:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:22:07.014913 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee45a37-fb1b-4d8d-8aa8-8c8ab39f8f54 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:22:07.016602 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-8ee45a37-fb1b-4d8d-8aa8-8c8ab39f8f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:07.029872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee45a37-fb1b-4d8d-8aa8-8c8ab39f8f54 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:22:07.033332 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f78d2b-efc8-4204-a9b4-f3a89365280e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:22:07.033961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f78d2b-efc8-4204-a9b4-f3a89365280e service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:07.034242 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f78d2b-efc8-4204-a9b4-f3a89365280e service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:07.034517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f78d2b-efc8-4204-a9b4-f3a89365280e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:07.043232 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f78d2b-efc8-4204-a9b4-f3a89365280e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:22:07.044681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f78d2b-efc8-4204-a9b4-f3a89365280e service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:22:07.054421 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 764/1539] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:22:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:22:16.694989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=11 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:17.049834 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=8 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:40.790383 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f18305-5610-4898-868c-a183542eb67c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:40.791347 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f18305-5610-4898-868c-a183542eb67c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:40.791347 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f18305-5610-4898-868c-a183542eb67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:40.827519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-63f18305-5610-4898-868c-a183542eb67c None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:40.831745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-63f18305-5610-4898-868c-a183542eb67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:40.852318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 776/1540] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:23:40.859352 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082219b-81bc-4fc4-bb81-6f1043d4ef6a None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:40.860536 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082219b-81bc-4fc4-bb81-6f1043d4ef6a None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:40.861064 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082219b-81bc-4fc4-bb81-6f1043d4ef6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:40.891163 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-8082219b-81bc-4fc4-bb81-6f1043d4ef6a None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:40.892687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-8082219b-81bc-4fc4-bb81-6f1043d4ef6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:40.912106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 765/1541] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:40.917132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f18305-5610-4898-868c-a183542eb67c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:40.934659 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f18305-5610-4898-868c-a183542eb67c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:40.950122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86dba7-8732-42c8-9b9a-5524a9519367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:40.950818 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86dba7-8732-42c8-9b9a-5524a9519367 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:40.951105 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86dba7-8732-42c8-9b9a-5524a9519367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:40.951563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86dba7-8732-42c8-9b9a-5524a9519367 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:40.952120 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86dba7-8732-42c8-9b9a-5524a9519367 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:40.952461 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86dba7-8732-42c8-9b9a-5524a9519367 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:40.954726 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86dba7-8732-42c8-9b9a-5524a9519367 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:40.971185 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 777/1542] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:40.980150 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082219b-81bc-4fc4-bb81-6f1043d4ef6a None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:40.996995 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082219b-81bc-4fc4-bb81-6f1043d4ef6a None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:41.006043 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd12d9c-9b56-4dd6-ae32-4ca2a9eb78b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:41.007251 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd12d9c-9b56-4dd6-ae32-4ca2a9eb78b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.007705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd12d9c-9b56-4dd6-ae32-4ca2a9eb78b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.008130 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd12d9c-9b56-4dd6-ae32-4ca2a9eb78b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.013705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd12d9c-9b56-4dd6-ae32-4ca2a9eb78b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:41.016545 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd12d9c-9b56-4dd6-ae32-4ca2a9eb78b1 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:41.065136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 766/1543] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:40 2022] POST /identity/v3/projects => generated 391 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:41.070027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86dba7-8732-42c8-9b9a-5524a9519367 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:41.083700 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86dba7-8732-42c8-9b9a-5524a9519367 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:41.099419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6811e35-7f5a-4f1d-9bb9-25bf424e9470 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.099654 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6811e35-7f5a-4f1d-9bb9-25bf424e9470 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.099897 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6811e35-7f5a-4f1d-9bb9-25bf424e9470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.138425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-e6811e35-7f5a-4f1d-9bb9-25bf424e9470 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:41.139673 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e6811e35-7f5a-4f1d-9bb9-25bf424e9470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:41.180138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2718825-cf75-4c0b-b593-f8369008f6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:41.181172 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2718825-cf75-4c0b-b593-f8369008f6fb admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.181631 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2718825-cf75-4c0b-b593-f8369008f6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.182247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2718825-cf75-4c0b-b593-f8369008f6fb admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.183147 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2718825-cf75-4c0b-b593-f8369008f6fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:41.190247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2718825-cf75-4c0b-b593-f8369008f6fb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:41.225277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 778/1544] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/users => generated 296 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:41.230850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2718825-cf75-4c0b-b593-f8369008f6fb admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:41.246856 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2718825-cf75-4c0b-b593-f8369008f6fb admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:41.259674 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa57d5b-c0aa-4998-ada2-d45c529f4708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:41.260834 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa57d5b-c0aa-4998-ada2-d45c529f4708 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.261458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa57d5b-c0aa-4998-ada2-d45c529f4708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.261967 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa57d5b-c0aa-4998-ada2-d45c529f4708 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.263320 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa57d5b-c0aa-4998-ada2-d45c529f4708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:41.265430 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa57d5b-c0aa-4998-ada2-d45c529f4708 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:41.269700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 767/1545] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:41.281437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26350bb9-2044-453d-9b20-64f4f1df692a None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.281997 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26350bb9-2044-453d-9b20-64f4f1df692a None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.282448 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26350bb9-2044-453d-9b20-64f4f1df692a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.283027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 779/1546] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:41.287952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa57d5b-c0aa-4998-ada2-d45c529f4708 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:41.304424 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa57d5b-c0aa-4998-ada2-d45c529f4708 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:41.312771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2c289-0e66-432d-be7b-729052b5be86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:41.314379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2c289-0e66-432d-be7b-729052b5be86 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.314913 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2c289-0e66-432d-be7b-729052b5be86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.315472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2c289-0e66-432d-be7b-729052b5be86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.316648 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2c289-0e66-432d-be7b-729052b5be86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:41.319104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2c289-0e66-432d-be7b-729052b5be86 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:41.325466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-26350bb9-2044-453d-9b20-64f4f1df692a None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:41.327041 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-26350bb9-2044-453d-9b20-64f4f1df692a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:41.342207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 780/1547] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:23:41.347234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2c289-0e66-432d-be7b-729052b5be86 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:41.361172 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2c289-0e66-432d-be7b-729052b5be86 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:41.366132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 768/1548] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:41.368726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26350bb9-2044-453d-9b20-64f4f1df692a None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:41.379414 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5764f-04f0-4072-bc05-7862016bf1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:41.382613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5764f-04f0-4072-bc05-7862016bf1a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.383393 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5764f-04f0-4072-bc05-7862016bf1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.384362 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5764f-04f0-4072-bc05-7862016bf1a2 admin admin] PATH_INFO: `/v3/projects/f0f63357734c4efa9a88c07996aa5551/users/cb06eeba738b452fab3986de7e1f7856/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.394748 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26350bb9-2044-453d-9b20-64f4f1df692a None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:41.404054 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5764f-04f0-4072-bc05-7862016bf1a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0f63357734c4efa9a88c07996aa5551, user_id=cb06eeba738b452fab3986de7e1f7856, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:41.405505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: WARNING py.warnings [None req-23c5764f-04f0-4072-bc05-7862016bf1a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:41.405505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: warnings.warn(msg) Jul 20 19:23:41.405505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: Jul 20 19:23:41.408341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5764f-04f0-4072-bc05-7862016bf1a2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:41.412097 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126181e-e29f-4af1-ac1b-23735937f1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:41.412843 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126181e-e29f-4af1-ac1b-23735937f1cd admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.413507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126181e-e29f-4af1-ac1b-23735937f1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.413507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126181e-e29f-4af1-ac1b-23735937f1cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.419696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126181e-e29f-4af1-ac1b-23735937f1cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:41.419696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126181e-e29f-4af1-ac1b-23735937f1cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:41.419696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: WARNING py.warnings [None req-4126181e-e29f-4af1-ac1b-23735937f1cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:41.419696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: warnings.warn(msg) Jul 20 19:23:41.419696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: Jul 20 19:23:41.419696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126181e-e29f-4af1-ac1b-23735937f1cd admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:41.460644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 769/1549] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:41.468623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 781/1550] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:41 2022] PUT /identity/v3/projects/f0f63357734c4efa9a88c07996aa5551/users/cb06eeba738b452fab3986de7e1f7856/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:41.472087 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126181e-e29f-4af1-ac1b-23735937f1cd admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:41.474750 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe509ed9-13b3-4352-a5d8-15d7820f2498 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.475323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe509ed9-13b3-4352-a5d8-15d7820f2498 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.475956 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe509ed9-13b3-4352-a5d8-15d7820f2498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.482074 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126181e-e29f-4af1-ac1b-23735937f1cd admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:41.517851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-fe509ed9-13b3-4352-a5d8-15d7820f2498 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:41.520991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fe509ed9-13b3-4352-a5d8-15d7820f2498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:41.570394 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9135e27-a986-4f88-a689-24c5cb5ccfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:41.570976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9135e27-a986-4f88-a689-24c5cb5ccfbd admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.571257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9135e27-a986-4f88-a689-24c5cb5ccfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.571530 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9135e27-a986-4f88-a689-24c5cb5ccfbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.572456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9135e27-a986-4f88-a689-24c5cb5ccfbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:41.574369 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9135e27-a986-4f88-a689-24c5cb5ccfbd admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:41.588810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 782/1551] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:41.594995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b122e5-a05c-4f05-b909-a3be7ee558dd None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.594995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b122e5-a05c-4f05-b909-a3be7ee558dd None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.594995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b122e5-a05c-4f05-b909-a3be7ee558dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.604735 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 770/1552] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:41.605910 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b985d-f3ab-4a17-9276-1a1aa201d637 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.606207 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b985d-f3ab-4a17-9276-1a1aa201d637 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.606529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b985d-f3ab-4a17-9276-1a1aa201d637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.618195 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b7b122e5-a05c-4f05-b909-a3be7ee558dd None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:41.619409 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b7b122e5-a05c-4f05-b909-a3be7ee558dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:41.623910 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-873b985d-f3ab-4a17-9276-1a1aa201d637 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:41.625139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-873b985d-f3ab-4a17-9276-1a1aa201d637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:41.639042 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:41.653517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:41.691016 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 783/1553] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:41.692731 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5db300-8598-45af-b90d-684a2b65f681 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.693132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5db300-8598-45af-b90d-684a2b65f681 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.693557 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5db300-8598-45af-b90d-684a2b65f681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.702283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 771/1554] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:41.702283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d4027-4b88-4ed0-9ebc-5ea4ac0064b5 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.702283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d4027-4b88-4ed0-9ebc-5ea4ac0064b5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.702283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d4027-4b88-4ed0-9ebc-5ea4ac0064b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.716945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-753d4027-4b88-4ed0-9ebc-5ea4ac0064b5 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:41.718156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-753d4027-4b88-4ed0-9ebc-5ea4ac0064b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:41.720145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2d5db300-8598-45af-b90d-684a2b65f681 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:41.721365 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2d5db300-8598-45af-b90d-684a2b65f681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:41.727113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 772/1555] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:41.728448 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89716b-3ad9-45af-81c4-38bb0f642e10 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.728794 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89716b-3ad9-45af-81c4-38bb0f642e10 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.729080 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89716b-3ad9-45af-81c4-38bb0f642e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.730294 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 784/1556] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:41.731404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82d471-a3b0-497f-b82a-4a811877bc7b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.731694 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82d471-a3b0-497f-b82a-4a811877bc7b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.731974 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82d471-a3b0-497f-b82a-4a811877bc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.750704 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-fd82d471-a3b0-497f-b82a-4a811877bc7b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:41.751946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fd82d471-a3b0-497f-b82a-4a811877bc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:41.752862 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-dd89716b-3ad9-45af-81c4-38bb0f642e10 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:41.754035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-dd89716b-3ad9-45af-81c4-38bb0f642e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:41.760328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 785/1557] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:41.761224 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82d471-a3b0-497f-b82a-4a811877bc7b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:41.769437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 773/1558] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:41.773237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89716b-3ad9-45af-81c4-38bb0f642e10 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:41.783810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89716b-3ad9-45af-81c4-38bb0f642e10 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:41.789526 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0594d-8db4-4e8c-b08c-3bcb3584fd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:41.790387 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0594d-8db4-4e8c-b08c-3bcb3584fd70 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.790615 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0594d-8db4-4e8c-b08c-3bcb3584fd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.790885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0594d-8db4-4e8c-b08c-3bcb3584fd70 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.791609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0594d-8db4-4e8c-b08c-3bcb3584fd70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:41.792646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: WARNING py.warnings [None req-c4a0594d-8db4-4e8c-b08c-3bcb3584fd70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:41.792646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: warnings.warn(msg) Jul 20 19:23:41.792646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: Jul 20 19:23:41.793841 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0594d-8db4-4e8c-b08c-3bcb3584fd70 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:41.820473 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 774/1559] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/users => generated 290 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:41.820473 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89339332-a03e-4fc7-be51-69401dbfd308 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.820473 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89339332-a03e-4fc7-be51-69401dbfd308 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.820473 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89339332-a03e-4fc7-be51-69401dbfd308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.825223 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82d471-a3b0-497f-b82a-4a811877bc7b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:41.829997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecccf6-2de2-4f4b-a38b-b81f7b60a744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:41.830743 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecccf6-2de2-4f4b-a38b-b81f7b60a744 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.831028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecccf6-2de2-4f4b-a38b-b81f7b60a744 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.831305 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecccf6-2de2-4f4b-a38b-b81f7b60a744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.838160 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-89339332-a03e-4fc7-be51-69401dbfd308 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:41.839746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-89339332-a03e-4fc7-be51-69401dbfd308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:41.842361 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fecccf6-2de2-4f4b-a38b-b81f7b60a744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:41.843810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fecccf6-2de2-4f4b-a38b-b81f7b60a744 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:41.849396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 775/1560] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:41.850393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4c6fc-8bc7-4645-9cec-e89f289b98d6 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.850680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4c6fc-8bc7-4645-9cec-e89f289b98d6 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.850966 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4c6fc-8bc7-4645-9cec-e89f289b98d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.860639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 786/1561] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:41.861797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1caff8-b205-403e-a3d1-e73963c2e569 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.862104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1caff8-b205-403e-a3d1-e73963c2e569 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.862371 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1caff8-b205-403e-a3d1-e73963c2e569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.869668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-aff4c6fc-8bc7-4645-9cec-e89f289b98d6 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:41.871060 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-aff4c6fc-8bc7-4645-9cec-e89f289b98d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:41.879664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 776/1562] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:41.881818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac74135-6513-4ef4-adb2-322e62ccac0c None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.881818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac74135-6513-4ef4-adb2-322e62ccac0c None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.882163 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac74135-6513-4ef4-adb2-322e62ccac0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.883423 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8f1caff8-b205-403e-a3d1-e73963c2e569 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:41.885614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8f1caff8-b205-403e-a3d1-e73963c2e569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:41.901093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 787/1563] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:41.904057 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7eaec-b8cc-4022-b6c3-f15e3271af56 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.904384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7eaec-b8cc-4022-b6c3-f15e3271af56 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.905067 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7eaec-b8cc-4022-b6c3-f15e3271af56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.906355 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-8ac74135-6513-4ef4-adb2-322e62ccac0c None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:41.907549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-8ac74135-6513-4ef4-adb2-322e62ccac0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:41.916672 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 777/1564] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:41.918047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62abdd-4336-47e0-b76c-68ba648d8c8e None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.918340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62abdd-4336-47e0-b76c-68ba648d8c8e None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.918619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62abdd-4336-47e0-b76c-68ba648d8c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.932366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-aec7eaec-b8cc-4022-b6c3-f15e3271af56 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:41.934238 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-aec7eaec-b8cc-4022-b6c3-f15e3271af56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:41.938475 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-cf62abdd-4336-47e0-b76c-68ba648d8c8e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:41.939652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-cf62abdd-4336-47e0-b76c-68ba648d8c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:41.946816 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 788/1565] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:41.948156 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7eaec-b8cc-4022-b6c3-f15e3271af56 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:41.953390 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 778/1566] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:41.957837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62abdd-4336-47e0-b76c-68ba648d8c8e None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:41.965888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7eaec-b8cc-4022-b6c3-f15e3271af56 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:41.969825 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62abdd-4336-47e0-b76c-68ba648d8c8e None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:41.971144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f833b89-214a-46d8-a190-63f2966be9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:41.972082 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f833b89-214a-46d8-a190-63f2966be9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.972419 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f833b89-214a-46d8-a190-63f2966be9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.972683 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f833b89-214a-46d8-a190-63f2966be9e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.973419 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f833b89-214a-46d8-a190-63f2966be9e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:41.975051 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6abd86-7ea0-438a-8031-02a5f4b8b734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:41.975181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f833b89-214a-46d8-a190-63f2966be9e4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:41.975720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6abd86-7ea0-438a-8031-02a5f4b8b734 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.976034 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6abd86-7ea0-438a-8031-02a5f4b8b734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.976400 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6abd86-7ea0-438a-8031-02a5f4b8b734 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.977139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6abd86-7ea0-438a-8031-02a5f4b8b734 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:41.977467 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6abd86-7ea0-438a-8031-02a5f4b8b734 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:41.979050 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6abd86-7ea0-438a-8031-02a5f4b8b734 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:41.987787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 789/1567] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:41.988691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f833b89-214a-46d8-a190-63f2966be9e4 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.002634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 779/1568] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:42.002634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6abd86-7ea0-438a-8031-02a5f4b8b734 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.002951 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f833b89-214a-46d8-a190-63f2966be9e4 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.008516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299abad1-433a-499b-9397-3a63df20d782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.008898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6abd86-7ea0-438a-8031-02a5f4b8b734 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.009253 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299abad1-433a-499b-9397-3a63df20d782 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.009545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299abad1-433a-499b-9397-3a63df20d782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.009828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299abad1-433a-499b-9397-3a63df20d782 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.010563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299abad1-433a-499b-9397-3a63df20d782 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:42.011318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299abad1-433a-499b-9397-3a63df20d782 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.012596 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299abad1-433a-499b-9397-3a63df20d782 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.014735 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba74da3-ad95-4125-a828-a0f802f2748e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.015577 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba74da3-ad95-4125-a828-a0f802f2748e admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.015926 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba74da3-ad95-4125-a828-a0f802f2748e admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.016278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba74da3-ad95-4125-a828-a0f802f2748e admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.017102 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba74da3-ad95-4125-a828-a0f802f2748e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:42.017480 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba74da3-ad95-4125-a828-a0f802f2748e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.019139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba74da3-ad95-4125-a828-a0f802f2748e admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.027352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 790/1569] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:42.029301 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299abad1-433a-499b-9397-3a63df20d782 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.038606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 780/1570] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:42.040144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299abad1-433a-499b-9397-3a63df20d782 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.041593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba74da3-ad95-4125-a828-a0f802f2748e admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.045696 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6b01b-aa87-4bff-b445-4620d853575d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.046605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6b01b-aa87-4bff-b445-4620d853575d admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.046942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6b01b-aa87-4bff-b445-4620d853575d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.047203 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6b01b-aa87-4bff-b445-4620d853575d admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.047940 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6b01b-aa87-4bff-b445-4620d853575d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:42.048245 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6b01b-aa87-4bff-b445-4620d853575d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.049891 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6b01b-aa87-4bff-b445-4620d853575d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.056164 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba74da3-ad95-4125-a828-a0f802f2748e admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.062335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86bd8a1-0091-4740-85ac-18177a5b7865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.063100 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86bd8a1-0091-4740-85ac-18177a5b7865 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.063448 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86bd8a1-0091-4740-85ac-18177a5b7865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.063921 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86bd8a1-0091-4740-85ac-18177a5b7865 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.064539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 791/1571] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:42.064978 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86bd8a1-0091-4740-85ac-18177a5b7865 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:42.065221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6b01b-aa87-4bff-b445-4620d853575d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.065684 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86bd8a1-0091-4740-85ac-18177a5b7865 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.067872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86bd8a1-0091-4740-85ac-18177a5b7865 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.076954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6b01b-aa87-4bff-b445-4620d853575d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.082694 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcf140-c93d-4806-89e8-f0dc8acfcfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.084175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcf140-c93d-4806-89e8-f0dc8acfcfdf admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.084392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcf140-c93d-4806-89e8-f0dc8acfcfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.084559 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcf140-c93d-4806-89e8-f0dc8acfcfdf admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.085313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcf140-c93d-4806-89e8-f0dc8acfcfdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:42.085594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcf140-c93d-4806-89e8-f0dc8acfcfdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.089456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcf140-c93d-4806-89e8-f0dc8acfcfdf admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.097374 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 781/1572] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:42.097374 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86bd8a1-0091-4740-85ac-18177a5b7865 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.106143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86bd8a1-0091-4740-85ac-18177a5b7865 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.112594 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a1c82-e4cd-450f-bc24-ffae3d1b740d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.113992 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a1c82-e4cd-450f-bc24-ffae3d1b740d admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.114767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a1c82-e4cd-450f-bc24-ffae3d1b740d admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.115261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a1c82-e4cd-450f-bc24-ffae3d1b740d admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.116316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a1c82-e4cd-450f-bc24-ffae3d1b740d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.116600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 792/1573] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:42.117422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcf140-c93d-4806-89e8-f0dc8acfcfdf admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.118664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a1c82-e4cd-450f-bc24-ffae3d1b740d admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.133140 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 782/1574] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:42.133992 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a1c82-e4cd-450f-bc24-ffae3d1b740d admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.135599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcf140-c93d-4806-89e8-f0dc8acfcfdf admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.140663 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8fe23-281c-45ea-af80-a42f425775f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.141672 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8fe23-281c-45ea-af80-a42f425775f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.141960 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8fe23-281c-45ea-af80-a42f425775f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.142240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8fe23-281c-45ea-af80-a42f425775f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.142943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8fe23-281c-45ea-af80-a42f425775f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.143988 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: WARNING py.warnings [None req-e7e8fe23-281c-45ea-af80-a42f425775f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:42.143988 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: warnings.warn(msg) Jul 20 19:23:42.143988 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: Jul 20 19:23:42.145219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8fe23-281c-45ea-af80-a42f425775f6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.149388 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a1c82-e4cd-450f-bc24-ffae3d1b740d admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.156141 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652b61b-a146-4007-8e6c-df2b5fde31e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.157248 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652b61b-a146-4007-8e6c-df2b5fde31e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.158119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652b61b-a146-4007-8e6c-df2b5fde31e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.158261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652b61b-a146-4007-8e6c-df2b5fde31e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.159227 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652b61b-a146-4007-8e6c-df2b5fde31e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.161452 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652b61b-a146-4007-8e6c-df2b5fde31e3 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.176554 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 793/1575] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:42 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:42.177647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8fe23-281c-45ea-af80-a42f425775f6 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.194262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 783/1576] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:42 2022] POST /identity/v3/projects => generated 363 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:42.195178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652b61b-a146-4007-8e6c-df2b5fde31e3 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.228555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:42.273145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8fe23-281c-45ea-af80-a42f425775f6 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.279301 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d4a3f-7128-4bb8-9089-b683f6e052a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.280192 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d4a3f-7128-4bb8-9089-b683f6e052a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.280574 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d4a3f-7128-4bb8-9089-b683f6e052a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.280922 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d4a3f-7128-4bb8-9089-b683f6e052a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.281575 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652b61b-a146-4007-8e6c-df2b5fde31e3 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.281713 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d4a3f-7128-4bb8-9089-b683f6e052a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.283966 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d4a3f-7128-4bb8-9089-b683f6e052a8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.287299 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e392e2-ddd4-4c71-bad6-97593dafc471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.288309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e392e2-ddd4-4c71-bad6-97593dafc471 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.288682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e392e2-ddd4-4c71-bad6-97593dafc471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.289282 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e392e2-ddd4-4c71-bad6-97593dafc471 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.289996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e392e2-ddd4-4c71-bad6-97593dafc471 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.292755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e392e2-ddd4-4c71-bad6-97593dafc471 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.297581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:42.318136 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 794/1577] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:42 2022] POST /identity/v3/projects => generated 357 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:42.319442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d4a3f-7128-4bb8-9089-b683f6e052a8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.339373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 784/1578] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:42 2022] POST /identity/v3/projects => generated 389 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:42.341232 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e392e2-ddd4-4c71-bad6-97593dafc471 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.406917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d4a3f-7128-4bb8-9089-b683f6e052a8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.452231 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e392e2-ddd4-4c71-bad6-97593dafc471 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.460622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c71dbb-03c9-4b89-8e97-25f89d5b18b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.461304 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c71dbb-03c9-4b89-8e97-25f89d5b18b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.462166 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c71dbb-03c9-4b89-8e97-25f89d5b18b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.462464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c71dbb-03c9-4b89-8e97-25f89d5b18b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.463220 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c71dbb-03c9-4b89-8e97-25f89d5b18b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.465095 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c71dbb-03c9-4b89-8e97-25f89d5b18b4 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.478045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bf6bf-787e-4dbe-b1e4-9af2e71e769e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.478725 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bf6bf-787e-4dbe-b1e4-9af2e71e769e admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.479023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bf6bf-787e-4dbe-b1e4-9af2e71e769e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.479276 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bf6bf-787e-4dbe-b1e4-9af2e71e769e admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.479986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bf6bf-787e-4dbe-b1e4-9af2e71e769e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.482007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bf6bf-787e-4dbe-b1e4-9af2e71e769e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.495779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 785/1579] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:42 2022] POST /identity/v3/projects => generated 405 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:42.497038 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c71dbb-03c9-4b89-8e97-25f89d5b18b4 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.514818 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 795/1580] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:42 2022] POST /identity/v3/projects => generated 375 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:42.515645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bf6bf-787e-4dbe-b1e4-9af2e71e769e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.574845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:42.583473 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c71dbb-03c9-4b89-8e97-25f89d5b18b4 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.597922 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bf6bf-787e-4dbe-b1e4-9af2e71e769e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.604477 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f5a20-b1f6-46c7-90c2-fd9312686b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.605171 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f5a20-b1f6-46c7-90c2-fd9312686b9c admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.605450 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f5a20-b1f6-46c7-90c2-fd9312686b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.605726 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f5a20-b1f6-46c7-90c2-fd9312686b9c admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.607040 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f5a20-b1f6-46c7-90c2-fd9312686b9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.609057 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f5a20-b1f6-46c7-90c2-fd9312686b9c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.641072 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 796/1581] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:42 2022] POST /identity/v3/users => generated 283 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:42.641473 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f5a20-b1f6-46c7-90c2-fd9312686b9c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.657199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f5a20-b1f6-46c7-90c2-fd9312686b9c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.667190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b54cd7-1e25-4414-a21f-169397051ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.667883 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b54cd7-1e25-4414-a21f-169397051ba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.668215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b54cd7-1e25-4414-a21f-169397051ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.668585 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b54cd7-1e25-4414-a21f-169397051ba1 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.669339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b54cd7-1e25-4414-a21f-169397051ba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.670737 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791090c-743a-43e8-88b6-8b306ae462d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.671569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b54cd7-1e25-4414-a21f-169397051ba1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.672109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791090c-743a-43e8-88b6-8b306ae462d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.672628 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791090c-743a-43e8-88b6-8b306ae462d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.673175 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791090c-743a-43e8-88b6-8b306ae462d5 admin admin] PATH_INFO: `/v3/projects/5231dfdf49c44f7ab191159ea563dad7/users/3b7b1014c466482b95820616b54dd13a/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.700386 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 797/1582] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:42 2022] POST /identity/v3/users => generated 282 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:42.702509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b54cd7-1e25-4414-a21f-169397051ba1 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.712624 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7791090c-743a-43e8-88b6-8b306ae462d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5231dfdf49c44f7ab191159ea563dad7, user_id=3b7b1014c466482b95820616b54dd13a, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.715331 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7791090c-743a-43e8-88b6-8b306ae462d5 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.726941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:42.727109 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b54cd7-1e25-4414-a21f-169397051ba1 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.734337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 786/1583] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:42 2022] PUT /identity/v3/projects/5231dfdf49c44f7ab191159ea563dad7/users/3b7b1014c466482b95820616b54dd13a/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:42.735675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791090c-743a-43e8-88b6-8b306ae462d5 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.802379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ba763-4394-43de-b640-d9d328e0d553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.803446 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ba763-4394-43de-b640-d9d328e0d553 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.803737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ba763-4394-43de-b640-d9d328e0d553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.804012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ba763-4394-43de-b640-d9d328e0d553 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.804811 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ba763-4394-43de-b640-d9d328e0d553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.806800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ba763-4394-43de-b640-d9d328e0d553 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.814061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791090c-743a-43e8-88b6-8b306ae462d5 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.819966 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd2e58-8cd7-4016-b96f-2f5e872d56ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.820993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd2e58-8cd7-4016-b96f-2f5e872d56ca admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.821288 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd2e58-8cd7-4016-b96f-2f5e872d56ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.821551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd2e58-8cd7-4016-b96f-2f5e872d56ca admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.822503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd2e58-8cd7-4016-b96f-2f5e872d56ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.824537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd2e58-8cd7-4016-b96f-2f5e872d56ca admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.831141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 798/1584] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:42 2022] POST /identity/v3/users => generated 279 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:42.834772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ba763-4394-43de-b640-d9d328e0d553 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.846852 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ba763-4394-43de-b640-d9d328e0d553 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.852675 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ef827-26f0-4fa3-9fd4-a7961b2ada67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.854141 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 787/1585] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:42 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:42.854681 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ef827-26f0-4fa3-9fd4-a7961b2ada67 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.855114 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd2e58-8cd7-4016-b96f-2f5e872d56ca admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.855456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ef827-26f0-4fa3-9fd4-a7961b2ada67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.855945 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ef827-26f0-4fa3-9fd4-a7961b2ada67 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.857077 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ef827-26f0-4fa3-9fd4-a7961b2ada67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.859324 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ef827-26f0-4fa3-9fd4-a7961b2ada67 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.866784 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd2e58-8cd7-4016-b96f-2f5e872d56ca admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.874622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ae492-ca9b-47f1-9bc6-77f54388a5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.875345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ae492-ca9b-47f1-9bc6-77f54388a5da admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.875621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ae492-ca9b-47f1-9bc6-77f54388a5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.875880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ae492-ca9b-47f1-9bc6-77f54388a5da admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.876836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ae492-ca9b-47f1-9bc6-77f54388a5da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.878933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ae492-ca9b-47f1-9bc6-77f54388a5da admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.889885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 799/1586] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:42 2022] POST /identity/v3/users => generated 303 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:42.891477 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ef827-26f0-4fa3-9fd4-a7961b2ada67 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.904155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ef827-26f0-4fa3-9fd4-a7961b2ada67 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.908536 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 788/1587] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:42 2022] POST /identity/v3/users => generated 288 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:42.910734 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ae492-ca9b-47f1-9bc6-77f54388a5da admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.913659 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277a921-26d4-4acb-bcc4-7b4f94ce946d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.914342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277a921-26d4-4acb-bcc4-7b4f94ce946d admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.914678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277a921-26d4-4acb-bcc4-7b4f94ce946d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.914957 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277a921-26d4-4acb-bcc4-7b4f94ce946d admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.915880 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277a921-26d4-4acb-bcc4-7b4f94ce946d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.917543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277a921-26d4-4acb-bcc4-7b4f94ce946d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.927357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ae492-ca9b-47f1-9bc6-77f54388a5da admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.932826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942a567-9fc8-497f-9ce7-7de67ac27e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.933501 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942a567-9fc8-497f-9ce7-7de67ac27e36 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.933942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942a567-9fc8-497f-9ce7-7de67ac27e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.934293 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942a567-9fc8-497f-9ce7-7de67ac27e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.935053 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942a567-9fc8-497f-9ce7-7de67ac27e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.937369 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942a567-9fc8-497f-9ce7-7de67ac27e36 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.940987 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 800/1588] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:42.944897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0519ac-255b-44b8-96cc-df9e6d904777 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.945656 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0519ac-255b-44b8-96cc-df9e6d904777 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.946298 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0519ac-255b-44b8-96cc-df9e6d904777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.955335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 789/1589] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:42.955922 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942a567-9fc8-497f-9ce7-7de67ac27e36 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.969816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942a567-9fc8-497f-9ce7-7de67ac27e36 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.974322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1f0519ac-255b-44b8-96cc-df9e6d904777 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:42.974781 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082e1e0-1752-4c9e-9445-5670f04f9a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.976266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082e1e0-1752-4c9e-9445-5670f04f9a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.976567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1f0519ac-255b-44b8-96cc-df9e6d904777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:42.976678 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082e1e0-1752-4c9e-9445-5670f04f9a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.976678 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082e1e0-1752-4c9e-9445-5670f04f9a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.977199 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082e1e0-1752-4c9e-9445-5670f04f9a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.979349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082e1e0-1752-4c9e-9445-5670f04f9a67 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.984600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 801/1590] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:42.989963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0519ac-255b-44b8-96cc-df9e6d904777 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.995584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 790/1591] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:42.995584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082e1e0-1752-4c9e-9445-5670f04f9a67 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.003382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0519ac-255b-44b8-96cc-df9e6d904777 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.006034 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082e1e0-1752-4c9e-9445-5670f04f9a67 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.010582 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f2dcb-8d3b-4f22-8c8c-db73915b1be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.011387 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f2dcb-8d3b-4f22-8c8c-db73915b1be1 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.011597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27691291-2afd-4783-8663-6d66e6a0a19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.011990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f2dcb-8d3b-4f22-8c8c-db73915b1be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.012279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27691291-2afd-4783-8663-6d66e6a0a19a admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.012488 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f2dcb-8d3b-4f22-8c8c-db73915b1be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.012603 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27691291-2afd-4783-8663-6d66e6a0a19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.012862 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27691291-2afd-4783-8663-6d66e6a0a19a admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.013604 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27691291-2afd-4783-8663-6d66e6a0a19a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.013964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f2dcb-8d3b-4f22-8c8c-db73915b1be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.015252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27691291-2afd-4783-8663-6d66e6a0a19a admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.016256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f2dcb-8d3b-4f22-8c8c-db73915b1be1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.027353 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 791/1592] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:43.028617 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27691291-2afd-4783-8663-6d66e6a0a19a admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.033949 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 802/1593] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:43.034900 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f2dcb-8d3b-4f22-8c8c-db73915b1be1 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.043136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27691291-2afd-4783-8663-6d66e6a0a19a admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.048080 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe9b75-f25c-4740-a7e9-8b46f5b6b2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.048793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe9b75-f25c-4740-a7e9-8b46f5b6b2bd admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.049140 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe9b75-f25c-4740-a7e9-8b46f5b6b2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.049424 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe9b75-f25c-4740-a7e9-8b46f5b6b2bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.050161 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe9b75-f25c-4740-a7e9-8b46f5b6b2bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.052361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe9b75-f25c-4740-a7e9-8b46f5b6b2bd admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.059718 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f2dcb-8d3b-4f22-8c8c-db73915b1be1 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.065280 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 792/1594] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:43.066041 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe9b75-f25c-4740-a7e9-8b46f5b6b2bd admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.068892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e0002-6da4-420f-a740-cae97732dc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.078113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e0002-6da4-420f-a740-cae97732dc2b admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.078444 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e0002-6da4-420f-a740-cae97732dc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.078690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e0002-6da4-420f-a740-cae97732dc2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.079473 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e0002-6da4-420f-a740-cae97732dc2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.082116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e0002-6da4-420f-a740-cae97732dc2b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.090444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe9b75-f25c-4740-a7e9-8b46f5b6b2bd admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.095230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 803/1595] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:43.095899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e0002-6da4-420f-a740-cae97732dc2b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.097133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d1541-82da-44a5-982b-1a14681847e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.097630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-830e0002-6da4-420f-a740-cae97732dc2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:43.097764 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d1541-82da-44a5-982b-1a14681847e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.098030 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d1541-82da-44a5-982b-1a14681847e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.099305 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d1541-82da-44a5-982b-1a14681847e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.100180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d1541-82da-44a5-982b-1a14681847e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.102204 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d1541-82da-44a5-982b-1a14681847e1 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.119697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 793/1596] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:43.120687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d1541-82da-44a5-982b-1a14681847e1 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.135352 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d1541-82da-44a5-982b-1a14681847e1 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.141803 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46426693-7f52-47f2-a2e8-420fd1a7c127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.142693 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46426693-7f52-47f2-a2e8-420fd1a7c127 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.142986 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46426693-7f52-47f2-a2e8-420fd1a7c127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.143266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46426693-7f52-47f2-a2e8-420fd1a7c127 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.144487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46426693-7f52-47f2-a2e8-420fd1a7c127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.147349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46426693-7f52-47f2-a2e8-420fd1a7c127 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.168201 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 794/1597] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:43.171244 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46426693-7f52-47f2-a2e8-420fd1a7c127 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.183422 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46426693-7f52-47f2-a2e8-420fd1a7c127 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.189288 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322a32e-3a26-4878-8fe7-a12da1eb532e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.190141 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322a32e-3a26-4878-8fe7-a12da1eb532e admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.190433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322a32e-3a26-4878-8fe7-a12da1eb532e admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.190719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322a32e-3a26-4878-8fe7-a12da1eb532e admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.191439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322a32e-3a26-4878-8fe7-a12da1eb532e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.193795 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322a32e-3a26-4878-8fe7-a12da1eb532e admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.196553 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e0002-6da4-420f-a740-cae97732dc2b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.209137 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 795/1598] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:43.210249 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322a32e-3a26-4878-8fe7-a12da1eb532e admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.218084 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ab51b-d519-4aba-9efe-1e46c39d7867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.220208 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ab51b-d519-4aba-9efe-1e46c39d7867 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.221066 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322a32e-3a26-4878-8fe7-a12da1eb532e admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.221570 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ab51b-d519-4aba-9efe-1e46c39d7867 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.223373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ab51b-d519-4aba-9efe-1e46c39d7867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.226883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75d26e-8571-46e4-b322-b45fe8d56ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.227850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75d26e-8571-46e4-b322-b45fe8d56ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.228115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75d26e-8571-46e4-b322-b45fe8d56ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.228494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75d26e-8571-46e4-b322-b45fe8d56ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.229380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75d26e-8571-46e4-b322-b45fe8d56ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.233058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75d26e-8571-46e4-b322-b45fe8d56ea1 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.244875 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:43.250616 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ab51b-d519-4aba-9efe-1e46c39d7867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.254944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 796/1599] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:43.254944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75d26e-8571-46e4-b322-b45fe8d56ea1 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.257521 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ab51b-d519-4aba-9efe-1e46c39d7867 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.266302 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75d26e-8571-46e4-b322-b45fe8d56ea1 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.271756 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 804/1600] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:43.272944 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ab51b-d519-4aba-9efe-1e46c39d7867 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.284470 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9049d-08ba-4c7b-9302-dd7679ea3bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.285061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ab51b-d519-4aba-9efe-1e46c39d7867 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.285849 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9049d-08ba-4c7b-9302-dd7679ea3bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.286259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9049d-08ba-4c7b-9302-dd7679ea3bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.287653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9049d-08ba-4c7b-9302-dd7679ea3bd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.288532 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9049d-08ba-4c7b-9302-dd7679ea3bd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.292718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9049d-08ba-4c7b-9302-dd7679ea3bd1 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.293034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e13fb-f1db-476c-9770-e4617547059a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.293955 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e13fb-f1db-476c-9770-e4617547059a admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.294238 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e13fb-f1db-476c-9770-e4617547059a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.294515 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e13fb-f1db-476c-9770-e4617547059a admin admin] PATH_INFO: `/v3/projects/cb8ace75ec1e4e73bfc1f2c37328d733/users/848d6cc392fd4d5680b814eddf730424/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.316436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e13fb-f1db-476c-9770-e4617547059a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb8ace75ec1e4e73bfc1f2c37328d733, user_id=848d6cc392fd4d5680b814eddf730424, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.320140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e13fb-f1db-476c-9770-e4617547059a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.320397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 797/1601] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:43.321269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9049d-08ba-4c7b-9302-dd7679ea3bd1 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.341842 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 805/1602] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:43 2022] PUT /identity/v3/projects/cb8ace75ec1e4e73bfc1f2c37328d733/users/848d6cc392fd4d5680b814eddf730424/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:43.343743 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e13fb-f1db-476c-9770-e4617547059a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.355046 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9049d-08ba-4c7b-9302-dd7679ea3bd1 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.427919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:43.463345 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e13fb-f1db-476c-9770-e4617547059a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.475925 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075e01a-6ac2-4098-998b-ec2c95376ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.478048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075e01a-6ac2-4098-998b-ec2c95376ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.478781 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075e01a-6ac2-4098-998b-ec2c95376ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.479794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075e01a-6ac2-4098-998b-ec2c95376ccf admin admin] PATH_INFO: `/v3/projects/b93ce1e7a02a4f8bb16d03d8c32c17bb/users/5e1a1407ccb448b986dec140237f4b8d/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.491097 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1edb79-b9a5-4977-9e47-6ffa7583ec68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.493899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1edb79-b9a5-4977-9e47-6ffa7583ec68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.496026 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1edb79-b9a5-4977-9e47-6ffa7583ec68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.496500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1edb79-b9a5-4977-9e47-6ffa7583ec68 admin admin] PATH_INFO: `/v3/projects/4e6e5965bb1e4d8c8577134ce542a311/users/3fc7d5e916d54b61b0bd38c60634c52b/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.517384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075e01a-6ac2-4098-998b-ec2c95376ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b93ce1e7a02a4f8bb16d03d8c32c17bb, user_id=5e1a1407ccb448b986dec140237f4b8d, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.524899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075e01a-6ac2-4098-998b-ec2c95376ccf admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.531721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1edb79-b9a5-4977-9e47-6ffa7583ec68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e6e5965bb1e4d8c8577134ce542a311, user_id=3fc7d5e916d54b61b0bd38c60634c52b, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.538068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1edb79-b9a5-4977-9e47-6ffa7583ec68 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.551426 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 806/1603] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:43 2022] PUT /identity/v3/projects/b93ce1e7a02a4f8bb16d03d8c32c17bb/users/5e1a1407ccb448b986dec140237f4b8d/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:43.552973 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075e01a-6ac2-4098-998b-ec2c95376ccf admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.571070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 798/1604] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:43 2022] PUT /identity/v3/projects/4e6e5965bb1e4d8c8577134ce542a311/users/3fc7d5e916d54b61b0bd38c60634c52b/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:43.587287 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1edb79-b9a5-4977-9e47-6ffa7583ec68 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.691145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1edb79-b9a5-4977-9e47-6ffa7583ec68 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.696497 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efebeaf-bf6c-48af-9b0c-edf60ffea123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.697410 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebeaf-bf6c-48af-9b0c-edf60ffea123 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.697696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebeaf-bf6c-48af-9b0c-edf60ffea123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.697976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebeaf-bf6c-48af-9b0c-edf60ffea123 admin admin] PATH_INFO: `/v3/projects/ff9c443652a141d482130869ff76c327/users/9ac790383bdf4eecaef96cc25fe38553/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.725889 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075e01a-6ac2-4098-998b-ec2c95376ccf admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.731854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efebeaf-bf6c-48af-9b0c-edf60ffea123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff9c443652a141d482130869ff76c327, user_id=9ac790383bdf4eecaef96cc25fe38553, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.736517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efebeaf-bf6c-48af-9b0c-edf60ffea123 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.750993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bb85b-163a-4f9f-88b9-30984e2d3171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.752062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bb85b-163a-4f9f-88b9-30984e2d3171 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.752541 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bb85b-163a-4f9f-88b9-30984e2d3171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.752936 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bb85b-163a-4f9f-88b9-30984e2d3171 admin admin] PATH_INFO: `/v3/projects/ad44b110b66740669f1360228ac3e901/users/434a0e5365ad4af19201dd3f331e6a34/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.782040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 799/1605] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:43 2022] PUT /identity/v3/projects/ff9c443652a141d482130869ff76c327/users/9ac790383bdf4eecaef96cc25fe38553/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:43.782040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efebeaf-bf6c-48af-9b0c-edf60ffea123 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.810216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130bb85b-163a-4f9f-88b9-30984e2d3171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad44b110b66740669f1360228ac3e901, user_id=434a0e5365ad4af19201dd3f331e6a34, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.818616 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130bb85b-163a-4f9f-88b9-30984e2d3171 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.887012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 807/1606] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:43 2022] PUT /identity/v3/projects/ad44b110b66740669f1360228ac3e901/users/434a0e5365ad4af19201dd3f331e6a34/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:43.887012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e06f2-513e-40fb-98d1-a1e30ab79cdb None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.887012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e06f2-513e-40fb-98d1-a1e30ab79cdb None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.887012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e06f2-513e-40fb-98d1-a1e30ab79cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.967535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2e8e06f2-513e-40fb-98d1-a1e30ab79cdb None None] MFA Rules not processed for user `848d6cc392fd4d5680b814eddf730424`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:43.968940 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2e8e06f2-513e-40fb-98d1-a1e30ab79cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:43.984373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efebeaf-bf6c-48af-9b0c-edf60ffea123 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:44.022034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:44.088964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:44.125882 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4232f-5f9e-481d-8b64-5ee9c793ab1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:44.127418 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4232f-5f9e-481d-8b64-5ee9c793ab1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:44.127908 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4232f-5f9e-481d-8b64-5ee9c793ab1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:44.128592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4232f-5f9e-481d-8b64-5ee9c793ab1f admin admin] PATH_INFO: `/v3/projects/654a322fff99416595c66a993aeaee17/users/b15f8d66d5104aaeae51e6c1fa3b3596/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:44.147854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 808/1607] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:44.150042 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5b377-f387-4272-8b96-4f51fcba725f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:44.150548 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5b377-f387-4272-8b96-4f51fcba725f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:44.151382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5b377-f387-4272-8b96-4f51fcba725f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:44.158237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d4232f-5f9e-481d-8b64-5ee9c793ab1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=654a322fff99416595c66a993aeaee17, user_id=b15f8d66d5104aaeae51e6c1fa3b3596, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:44.162599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d4232f-5f9e-481d-8b64-5ee9c793ab1f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:44.201381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 800/1608] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:43 2022] PUT /identity/v3/projects/654a322fff99416595c66a993aeaee17/users/b15f8d66d5104aaeae51e6c1fa3b3596/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:44.202983 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b41b99-7f68-4615-92fd-293e5da0e8dd None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:44.203277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b41b99-7f68-4615-92fd-293e5da0e8dd None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:44.203727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b41b99-7f68-4615-92fd-293e5da0e8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:44.206607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-faa5b377-f387-4272-8b96-4f51fcba725f None None] MFA Rules not processed for user `5e1a1407ccb448b986dec140237f4b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:44.208895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-faa5b377-f387-4272-8b96-4f51fcba725f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:44.237429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-19b41b99-7f68-4615-92fd-293e5da0e8dd None None] MFA Rules not processed for user `3fc7d5e916d54b61b0bd38c60634c52b`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:44.239041 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-19b41b99-7f68-4615-92fd-293e5da0e8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:44.380094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 809/1609] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:44 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:44.380094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b695c-1015-48f3-bbf5-91c72bb73534 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:44.380094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b695c-1015-48f3-bbf5-91c72bb73534 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:44.380094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b695c-1015-48f3-bbf5-91c72bb73534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:44.402396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 801/1610] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:44 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:44.405884 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625ebff-3d7b-4a23-9eb5-43942e47838f None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:44.406183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625ebff-3d7b-4a23-9eb5-43942e47838f None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:44.406514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625ebff-3d7b-4a23-9eb5-43942e47838f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:44.408483 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-dd5b695c-1015-48f3-bbf5-91c72bb73534 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:44.409789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-dd5b695c-1015-48f3-bbf5-91c72bb73534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:44.454076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-1625ebff-3d7b-4a23-9eb5-43942e47838f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:44.457701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-1625ebff-3d7b-4a23-9eb5-43942e47838f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:44.570499 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 802/1611] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:44.571759 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625ebff-3d7b-4a23-9eb5-43942e47838f None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:44.585317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 810/1612] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:44.590113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f042ab-0806-4c06-98f3-a57a116b220e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:44.591010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625ebff-3d7b-4a23-9eb5-43942e47838f None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:44.593273 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f042ab-0806-4c06-98f3-a57a116b220e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:44.593708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f042ab-0806-4c06-98f3-a57a116b220e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:44.602436 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15e758-3610-453d-8a7e-edb2840ec951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:44.603483 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15e758-3610-453d-8a7e-edb2840ec951 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:44.604476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15e758-3610-453d-8a7e-edb2840ec951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:44.605072 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15e758-3610-453d-8a7e-edb2840ec951 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:44.606709 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15e758-3610-453d-8a7e-edb2840ec951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:44.609618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15e758-3610-453d-8a7e-edb2840ec951 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:44.644048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b4f042ab-0806-4c06-98f3-a57a116b220e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:44.647070 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b4f042ab-0806-4c06-98f3-a57a116b220e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:44.651024 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 803/1613] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:44 2022] POST /identity/v3/projects => generated 361 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:44.652191 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15e758-3610-453d-8a7e-edb2840ec951 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:44.726805 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:44.753436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:44.785765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15e758-3610-453d-8a7e-edb2840ec951 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:44.790966 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 811/1614] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:44.799716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f042ab-0806-4c06-98f3-a57a116b220e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:44.809315 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27677e-0ea5-4f17-91d3-5f789e898a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:44.809693 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27677e-0ea5-4f17-91d3-5f789e898a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:44.809928 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27677e-0ea5-4f17-91d3-5f789e898a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:44.810476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27677e-0ea5-4f17-91d3-5f789e898a2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:44.811629 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da27677e-0ea5-4f17-91d3-5f789e898a2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:44.817018 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da27677e-0ea5-4f17-91d3-5f789e898a2a admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:44.822213 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f042ab-0806-4c06-98f3-a57a116b220e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:44.834741 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276f363-66aa-4ce5-8894-2b2d0fa50483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:44.835776 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276f363-66aa-4ce5-8894-2b2d0fa50483 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:44.835776 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276f363-66aa-4ce5-8894-2b2d0fa50483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:44.836085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276f363-66aa-4ce5-8894-2b2d0fa50483 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:44.836743 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276f363-66aa-4ce5-8894-2b2d0fa50483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:44.839780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276f363-66aa-4ce5-8894-2b2d0fa50483 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:44.876530 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 804/1615] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:44 2022] POST /identity/v3/projects => generated 353 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:44.877474 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27677e-0ea5-4f17-91d3-5f789e898a2a admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:44.904190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 812/1616] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:44 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:44.904190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276f363-66aa-4ce5-8894-2b2d0fa50483 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.011434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276f363-66aa-4ce5-8894-2b2d0fa50483 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.021797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f6c47-c95b-4ce2-b526-56476513cf4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.022950 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f6c47-c95b-4ce2-b526-56476513cf4f service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.023241 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f6c47-c95b-4ce2-b526-56476513cf4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.024639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27677e-0ea5-4f17-91d3-5f789e898a2a admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.025262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f6c47-c95b-4ce2-b526-56476513cf4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.033180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa8c6a-47aa-4d11-b28c-ce84cfe5ccfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.034531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa8c6a-47aa-4d11-b28c-ce84cfe5ccfc service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.045146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa8c6a-47aa-4d11-b28c-ce84cfe5ccfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.045146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa8c6a-47aa-4d11-b28c-ce84cfe5ccfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.155293 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa8c6a-47aa-4d11-b28c-ce84cfe5ccfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.157418 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa8c6a-47aa-4d11-b28c-ce84cfe5ccfc service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.174631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 805/1617] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:45.175968 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa8c6a-47aa-4d11-b28c-ce84cfe5ccfc service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.185366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f6c47-c95b-4ce2-b526-56476513cf4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.187022 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f6c47-c95b-4ce2-b526-56476513cf4f service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.189865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa8c6a-47aa-4d11-b28c-ce84cfe5ccfc service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.197361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45486244-13f6-43a0-a2bd-c9b2342360a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.198453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45486244-13f6-43a0-a2bd-c9b2342360a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.198975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45486244-13f6-43a0-a2bd-c9b2342360a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.199482 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45486244-13f6-43a0-a2bd-c9b2342360a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.201280 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45486244-13f6-43a0-a2bd-c9b2342360a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.207605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45486244-13f6-43a0-a2bd-c9b2342360a5 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.213286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 813/1618] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:45.214824 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f6c47-c95b-4ce2-b526-56476513cf4f service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.232409 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f6c47-c95b-4ce2-b526-56476513cf4f service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.236393 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77cf9c0-f1cf-4bd7-b7d3-ff55e0b8b400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.237053 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cf9c0-f1cf-4bd7-b7d3-ff55e0b8b400 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.237338 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cf9c0-f1cf-4bd7-b7d3-ff55e0b8b400 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.237611 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cf9c0-f1cf-4bd7-b7d3-ff55e0b8b400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.241972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 806/1619] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:45 2022] POST /identity/v3/users => generated 287 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:45.243180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45486244-13f6-43a0-a2bd-c9b2342360a5 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.253113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77cf9c0-f1cf-4bd7-b7d3-ff55e0b8b400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.254649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77cf9c0-f1cf-4bd7-b7d3-ff55e0b8b400 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.268085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 814/1620] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:45.269259 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77cf9c0-f1cf-4bd7-b7d3-ff55e0b8b400 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.274860 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45486244-13f6-43a0-a2bd-c9b2342360a5 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.284266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77cf9c0-f1cf-4bd7-b7d3-ff55e0b8b400 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.291410 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8ccd2-0a5f-4d8f-b085-85518e8c7fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.293150 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc93898-1885-42db-a3b9-13a3d57daccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.293472 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8ccd2-0a5f-4d8f-b085-85518e8c7fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.293885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc93898-1885-42db-a3b9-13a3d57daccb admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.294171 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc93898-1885-42db-a3b9-13a3d57daccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.294445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8ccd2-0a5f-4d8f-b085-85518e8c7fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.294901 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8ccd2-0a5f-4d8f-b085-85518e8c7fb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.295597 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc93898-1885-42db-a3b9-13a3d57daccb admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.295723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8ccd2-0a5f-4d8f-b085-85518e8c7fb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.296636 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc93898-1885-42db-a3b9-13a3d57daccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.299064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc93898-1885-42db-a3b9-13a3d57daccb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.301774 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8ccd2-0a5f-4d8f-b085-85518e8c7fb0 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.338041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 815/1621] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:45 2022] POST /identity/v3/users => generated 289 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:45.338041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc93898-1885-42db-a3b9-13a3d57daccb admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.352472 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 807/1622] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:45 2022] POST /identity/v3/users => generated 283 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:45.353425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8ccd2-0a5f-4d8f-b085-85518e8c7fb0 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.366266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc93898-1885-42db-a3b9-13a3d57daccb admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.367137 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8ccd2-0a5f-4d8f-b085-85518e8c7fb0 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.373543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaaf37-47d1-4cf0-b469-11c8bbbf4e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.376188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaaf37-47d1-4cf0-b469-11c8bbbf4e95 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.376612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaaf37-47d1-4cf0-b469-11c8bbbf4e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.376797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaaf37-47d1-4cf0-b469-11c8bbbf4e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.378311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaaf37-47d1-4cf0-b469-11c8bbbf4e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.378808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddeeb6-2bda-46c0-9fbc-0faf4bbf9aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.380495 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddeeb6-2bda-46c0-9fbc-0faf4bbf9aae admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.380842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaaf37-47d1-4cf0-b469-11c8bbbf4e95 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.393033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddeeb6-2bda-46c0-9fbc-0faf4bbf9aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.393329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddeeb6-2bda-46c0-9fbc-0faf4bbf9aae admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.394740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddeeb6-2bda-46c0-9fbc-0faf4bbf9aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.398922 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddeeb6-2bda-46c0-9fbc-0faf4bbf9aae admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.420604 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 808/1623] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:45.420604 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaaf37-47d1-4cf0-b469-11c8bbbf4e95 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.450574 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaaf37-47d1-4cf0-b469-11c8bbbf4e95 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.455758 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 816/1624] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:45.458991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddeeb6-2bda-46c0-9fbc-0faf4bbf9aae admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.469146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5418a9c-2df1-4f50-bab1-3f4be36a8ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.469847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5418a9c-2df1-4f50-bab1-3f4be36a8ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.470129 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5418a9c-2df1-4f50-bab1-3f4be36a8ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.470403 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5418a9c-2df1-4f50-bab1-3f4be36a8ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.471171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5418a9c-2df1-4f50-bab1-3f4be36a8ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.473015 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5418a9c-2df1-4f50-bab1-3f4be36a8ee9 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.503859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 809/1625] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:45.506011 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5418a9c-2df1-4f50-bab1-3f4be36a8ee9 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.507814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddeeb6-2bda-46c0-9fbc-0faf4bbf9aae admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.513206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e43de-3df0-4cb8-953f-d333b795218a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.514513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e43de-3df0-4cb8-953f-d333b795218a admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.515021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e43de-3df0-4cb8-953f-d333b795218a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.515524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e43de-3df0-4cb8-953f-d333b795218a admin admin] PATH_INFO: `/v3/projects/e6a6f6eb1aa84ffab9ec5117df9e0f5b/users/4d8cc7f472844daea4cfb6afa6d8db3e/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.531450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5418a9c-2df1-4f50-bab1-3f4be36a8ee9 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.542730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8410d-0b23-49f3-bb9f-754ae289e0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.545096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8410d-0b23-49f3-bb9f-754ae289e0af admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.547085 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8410d-0b23-49f3-bb9f-754ae289e0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.547476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8410d-0b23-49f3-bb9f-754ae289e0af admin admin] PATH_INFO: `/v3/projects/39dd51065ee3444fb808ac75eac7973a/users/31cca6ef8541472a8a10eae2b345b6b2/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.549558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e43de-3df0-4cb8-953f-d333b795218a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a6f6eb1aa84ffab9ec5117df9e0f5b, user_id=4d8cc7f472844daea4cfb6afa6d8db3e, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.556691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e43de-3df0-4cb8-953f-d333b795218a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.581530 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:45.591850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8410d-0b23-49f3-bb9f-754ae289e0af admin admin] RBAC: Authorizing `identity:create_grant(project_id=39dd51065ee3444fb808ac75eac7973a, user_id=31cca6ef8541472a8a10eae2b345b6b2, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.592673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 817/1626] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:45 2022] PUT /identity/v3/projects/e6a6f6eb1aa84ffab9ec5117df9e0f5b/users/4d8cc7f472844daea4cfb6afa6d8db3e/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:45.597600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e43de-3df0-4cb8-953f-d333b795218a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.598595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8410d-0b23-49f3-bb9f-754ae289e0af admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.630410 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 810/1627] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:45 2022] PUT /identity/v3/projects/39dd51065ee3444fb808ac75eac7973a/users/31cca6ef8541472a8a10eae2b345b6b2/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:45.631486 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8410d-0b23-49f3-bb9f-754ae289e0af admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.641825 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:45.744511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e43de-3df0-4cb8-953f-d333b795218a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.791981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8410d-0b23-49f3-bb9f-754ae289e0af admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.806170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af77f2-afcf-4813-bafd-67ef34f5b9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.806999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af77f2-afcf-4813-bafd-67ef34f5b9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.807149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af77f2-afcf-4813-bafd-67ef34f5b9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.807408 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af77f2-afcf-4813-bafd-67ef34f5b9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.808214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af77f2-afcf-4813-bafd-67ef34f5b9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.810048 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af77f2-afcf-4813-bafd-67ef34f5b9f9 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.840024 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 811/1628] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:45.841430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af77f2-afcf-4813-bafd-67ef34f5b9f9 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.865178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af77f2-afcf-4813-bafd-67ef34f5b9f9 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.877100 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32622e5-ab8f-4d95-bc0c-0e416d3a5376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.877762 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32622e5-ab8f-4d95-bc0c-0e416d3a5376 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.878079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32622e5-ab8f-4d95-bc0c-0e416d3a5376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.878365 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32622e5-ab8f-4d95-bc0c-0e416d3a5376 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.880709 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32622e5-ab8f-4d95-bc0c-0e416d3a5376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.885439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32622e5-ab8f-4d95-bc0c-0e416d3a5376 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.904272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84337ae-e344-440a-9634-43475ca6ddff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.905845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84337ae-e344-440a-9634-43475ca6ddff admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.906346 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84337ae-e344-440a-9634-43475ca6ddff admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.906837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84337ae-e344-440a-9634-43475ca6ddff admin admin] PATH_INFO: `/v3/projects/f537f6c99e3b495a829fe6f82150c8d0/users/4371317787784e91b34df08a6b964ffd/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.909295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 812/1629] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:45.910431 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32622e5-ab8f-4d95-bc0c-0e416d3a5376 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.933230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84337ae-e344-440a-9634-43475ca6ddff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f537f6c99e3b495a829fe6f82150c8d0, user_id=4371317787784e91b34df08a6b964ffd, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.935815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32622e5-ab8f-4d95-bc0c-0e416d3a5376 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.937403 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84337ae-e344-440a-9634-43475ca6ddff admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.948144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a0bb1-0f1b-4d9c-b708-8084357b0447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.949024 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a0bb1-0f1b-4d9c-b708-8084357b0447 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.949312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a0bb1-0f1b-4d9c-b708-8084357b0447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.949654 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a0bb1-0f1b-4d9c-b708-8084357b0447 admin admin] PATH_INFO: `/v3/projects/e6a6f6eb1aa84ffab9ec5117df9e0f5b/users/4d8cc7f472844daea4cfb6afa6d8db3e/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.952475 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a0bb1-0f1b-4d9c-b708-8084357b0447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a6f6eb1aa84ffab9ec5117df9e0f5b, user_id=4d8cc7f472844daea4cfb6afa6d8db3e, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.956468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a0bb1-0f1b-4d9c-b708-8084357b0447 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.965113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 818/1630] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:45 2022] PUT /identity/v3/projects/f537f6c99e3b495a829fe6f82150c8d0/users/4371317787784e91b34df08a6b964ffd/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:45.966388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84337ae-e344-440a-9634-43475ca6ddff admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.979177 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 813/1631] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:45 2022] PUT /identity/v3/projects/e6a6f6eb1aa84ffab9ec5117df9e0f5b/users/4d8cc7f472844daea4cfb6afa6d8db3e/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:45.979177 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a0bb1-0f1b-4d9c-b708-8084357b0447 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:46.140704 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84337ae-e344-440a-9634-43475ca6ddff admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:46.143450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a0bb1-0f1b-4d9c-b708-8084357b0447 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:46.152963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f86fb-dd4f-4436-9cca-3f1e6e17288d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:46.156027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbfa4a-f19a-40b2-9470-4d7aa43d11e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:46.156222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f86fb-dd4f-4436-9cca-3f1e6e17288d admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:46.156996 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f86fb-dd4f-4436-9cca-3f1e6e17288d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:46.157923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f86fb-dd4f-4436-9cca-3f1e6e17288d admin admin] PATH_INFO: `/v3/projects/39dd51065ee3444fb808ac75eac7973a/users/31cca6ef8541472a8a10eae2b345b6b2/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:46.159859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbfa4a-f19a-40b2-9470-4d7aa43d11e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:46.159859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbfa4a-f19a-40b2-9470-4d7aa43d11e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:46.159859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbfa4a-f19a-40b2-9470-4d7aa43d11e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:46.161958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbfa4a-f19a-40b2-9470-4d7aa43d11e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:46.164652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f86fb-dd4f-4436-9cca-3f1e6e17288d admin admin] RBAC: Authorizing `identity:create_grant(project_id=39dd51065ee3444fb808ac75eac7973a, user_id=31cca6ef8541472a8a10eae2b345b6b2, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:46.166275 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbfa4a-f19a-40b2-9470-4d7aa43d11e3 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:46.170999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f86fb-dd4f-4436-9cca-3f1e6e17288d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:46.201044 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 814/1632] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:46.204621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb62860-1573-4481-b723-b64b9b192237 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:46.205748 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb62860-1573-4481-b723-b64b9b192237 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:46.206133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb62860-1573-4481-b723-b64b9b192237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:46.218840 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 819/1633] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:45 2022] PUT /identity/v3/projects/39dd51065ee3444fb808ac75eac7973a/users/31cca6ef8541472a8a10eae2b345b6b2/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:46.218840 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f86fb-dd4f-4436-9cca-3f1e6e17288d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:46.278169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-eeb62860-1573-4481-b723-b64b9b192237 None None] MFA Rules not processed for user `4d8cc7f472844daea4cfb6afa6d8db3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:46.281713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-eeb62860-1573-4481-b723-b64b9b192237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:46.346377 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:46.396400 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f86fb-dd4f-4436-9cca-3f1e6e17288d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:46.407477 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff791ef7-48e5-4960-8ef1-481a058a7a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:46.408876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff791ef7-48e5-4960-8ef1-481a058a7a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:46.411803 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff791ef7-48e5-4960-8ef1-481a058a7a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:46.411803 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff791ef7-48e5-4960-8ef1-481a058a7a6d admin admin] PATH_INFO: `/v3/projects/f537f6c99e3b495a829fe6f82150c8d0/users/4371317787784e91b34df08a6b964ffd/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:46.417092 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff791ef7-48e5-4960-8ef1-481a058a7a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f537f6c99e3b495a829fe6f82150c8d0, user_id=4371317787784e91b34df08a6b964ffd, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:46.422401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff791ef7-48e5-4960-8ef1-481a058a7a6d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:46.424347 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:46.471946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 820/1634] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:46 2022] PUT /identity/v3/projects/f537f6c99e3b495a829fe6f82150c8d0/users/4371317787784e91b34df08a6b964ffd/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:46.477385 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222d9e6-6696-4107-8289-7a1796401d04 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:46.477793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222d9e6-6696-4107-8289-7a1796401d04 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:46.478210 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222d9e6-6696-4107-8289-7a1796401d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:46.493267 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 815/1635] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:46 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:46.503214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674c25d-c67b-46b0-ab38-850c5bf9c22d None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:46.507042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674c25d-c67b-46b0-ab38-850c5bf9c22d None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:46.507042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674c25d-c67b-46b0-ab38-850c5bf9c22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:46.548977 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c222d9e6-6696-4107-8289-7a1796401d04 None None] MFA Rules not processed for user `31cca6ef8541472a8a10eae2b345b6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:46.551127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c222d9e6-6696-4107-8289-7a1796401d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:46.570811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-6674c25d-c67b-46b0-ab38-850c5bf9c22d None None] MFA Rules not processed for user `4371317787784e91b34df08a6b964ffd`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:46.572926 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6674c25d-c67b-46b0-ab38-850c5bf9c22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:46.746937 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 821/1636] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:46 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:46.748244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222d9e6-6696-4107-8289-7a1796401d04 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:46.773632 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 816/1637] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:46 2022] POST /identity/v3/auth/tokens => generated 3287 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:46.780976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674c25d-c67b-46b0-ab38-850c5bf9c22d None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:46.908211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222d9e6-6696-4107-8289-7a1796401d04 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:46.919634 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243d14a-1f35-426f-a71f-75a1a00de224 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:46.920321 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243d14a-1f35-426f-a71f-75a1a00de224 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:46.920608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243d14a-1f35-426f-a71f-75a1a00de224 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:46.920884 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243d14a-1f35-426f-a71f-75a1a00de224 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:46.933892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674c25d-c67b-46b0-ab38-850c5bf9c22d None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:46.946099 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d738c7-0921-46ef-98fb-dddc0ae24bdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:46.949054 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d738c7-0921-46ef-98fb-dddc0ae24bdb service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:46.950379 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d738c7-0921-46ef-98fb-dddc0ae24bdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:46.951950 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d738c7-0921-46ef-98fb-dddc0ae24bdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:46.977933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d738c7-0921-46ef-98fb-dddc0ae24bdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:46.979312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d738c7-0921-46ef-98fb-dddc0ae24bdb service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:46.996123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 817/1638] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:46 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:47.050710 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243d14a-1f35-426f-a71f-75a1a00de224 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:47.056887 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243d14a-1f35-426f-a71f-75a1a00de224 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:47.074729 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 822/1639] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:46 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:47.118839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d738c7-0921-46ef-98fb-dddc0ae24bdb service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:47.131795 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d738c7-0921-46ef-98fb-dddc0ae24bdb service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:47.144029 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205d279-801b-4688-bf44-d63575091c5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:47.146173 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205d279-801b-4688-bf44-d63575091c5f service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:47.146587 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205d279-801b-4688-bf44-d63575091c5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:47.146887 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205d279-801b-4688-bf44-d63575091c5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:47.158375 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a205d279-801b-4688-bf44-d63575091c5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:47.159918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a205d279-801b-4688-bf44-d63575091c5f service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:47.211702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 818/1640] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:47 2022] GET /identity/v3/auth/tokens => generated 3287 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:47.376142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243d14a-1f35-426f-a71f-75a1a00de224 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:47.401739 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243d14a-1f35-426f-a71f-75a1a00de224 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:47.429168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec45f6-3f5d-4945-896a-9057857c377c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:47.429951 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec45f6-3f5d-4945-896a-9057857c377c service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:47.430408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec45f6-3f5d-4945-896a-9057857c377c service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:47.430661 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec45f6-3f5d-4945-896a-9057857c377c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:47.450606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:47.470930 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205d279-801b-4688-bf44-d63575091c5f service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:47.504390 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205d279-801b-4688-bf44-d63575091c5f service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:47.512114 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1bb58-baa7-4b99-b731-216abf3cd5c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:47.513521 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1bb58-baa7-4b99-b731-216abf3cd5c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:47.514417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1bb58-baa7-4b99-b731-216abf3cd5c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:47.515021 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1bb58-baa7-4b99-b731-216abf3cd5c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:47.636416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec45f6-3f5d-4945-896a-9057857c377c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:47.638688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec45f6-3f5d-4945-896a-9057857c377c service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:47.661909 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 823/1641] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:47 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:47.733680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1bb58-baa7-4b99-b731-216abf3cd5c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:47.736468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1bb58-baa7-4b99-b731-216abf3cd5c2 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:47.747956 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:47.765893 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 819/1642] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:47 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:23:48.201175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda92b0b-6fe4-455b-a22d-ef167b23d361 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:48.208998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda92b0b-6fe4-455b-a22d-ef167b23d361 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:48.209301 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda92b0b-6fe4-455b-a22d-ef167b23d361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:48.293712 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-eda92b0b-6fe4-455b-a22d-ef167b23d361 None None] MFA Rules not processed for user `434a0e5365ad4af19201dd3f331e6a34`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:48.295823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-eda92b0b-6fe4-455b-a22d-ef167b23d361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:48.507834 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 824/1643] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:48 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:48.513777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1bb58-baa7-4b99-b731-216abf3cd5c2 service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:48.661721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1bb58-baa7-4b99-b731-216abf3cd5c2 service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:48.733295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2abff-2a6a-4ca5-bd3b-b096516ff622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:48.733295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2abff-2a6a-4ca5-bd3b-b096516ff622 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:48.733295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2abff-2a6a-4ca5-bd3b-b096516ff622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:48.733295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2abff-2a6a-4ca5-bd3b-b096516ff622 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:48.736998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2abff-2a6a-4ca5-bd3b-b096516ff622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:48.736998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2abff-2a6a-4ca5-bd3b-b096516ff622 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:48.740942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda92b0b-6fe4-455b-a22d-ef167b23d361 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:48.763795 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda92b0b-6fe4-455b-a22d-ef167b23d361 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:48.782480 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c02807-65de-4224-94b0-71e5ff7b57c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:48.784799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c02807-65de-4224-94b0-71e5ff7b57c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:48.788004 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c02807-65de-4224-94b0-71e5ff7b57c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:48.790677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c02807-65de-4224-94b0-71e5ff7b57c1 admin admin] PATH_INFO: `/v3/users/848d6cc392fd4d5680b814eddf730424` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:48.794162 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c02807-65de-4224-94b0-71e5ff7b57c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=848d6cc392fd4d5680b814eddf730424)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:48.798017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: WARNING py.warnings [None req-10c02807-65de-4224-94b0-71e5ff7b57c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:48.798017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: warnings.warn(msg) Jul 20 19:23:48.798017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: Jul 20 19:23:48.800650 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c02807-65de-4224-94b0-71e5ff7b57c1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:48.820586 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 820/1644] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:48 2022] POST /identity/v3/projects => generated 405 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:48.823014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2abff-2a6a-4ca5-bd3b-b096516ff622 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:49.013835 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2abff-2a6a-4ca5-bd3b-b096516ff622 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:49.033746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725aac9-752f-4a62-aa94-ac59eb9090cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:49.034488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725aac9-752f-4a62-aa94-ac59eb9090cc admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:49.034766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725aac9-752f-4a62-aa94-ac59eb9090cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:49.035265 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725aac9-752f-4a62-aa94-ac59eb9090cc admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:49.037287 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725aac9-752f-4a62-aa94-ac59eb9090cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:49.039001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725aac9-752f-4a62-aa94-ac59eb9090cc admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:49.052736 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-10c02807-65de-4224-94b0-71e5ff7b57c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '848d6cc392fd4d5680b814eddf730424'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:49.064178 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-10c02807-65de-4224-94b0-71e5ff7b57c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '848d6cc392fd4d5680b814eddf730424'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:49.067707 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-10c02807-65de-4224-94b0-71e5ff7b57c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '848d6cc392fd4d5680b814eddf730424'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:49.092520 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-10c02807-65de-4224-94b0-71e5ff7b57c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '848d6cc392fd4d5680b814eddf730424'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:49.097283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 821/1645] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:48 2022] POST /identity/v3/users => generated 309 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:49.105904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725aac9-752f-4a62-aa94-ac59eb9090cc admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:49.113172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c725aac9-752f-4a62-aa94-ac59eb9090cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:49.157249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-10c02807-65de-4224-94b0-71e5ff7b57c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '848d6cc392fd4d5680b814eddf730424'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:49.161094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725aac9-752f-4a62-aa94-ac59eb9090cc admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:49.179132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 825/1646] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:23:48 2022] DELETE /identity/v3/users/848d6cc392fd4d5680b814eddf730424 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:49.185195 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc625d8-c474-4c57-880d-51fb75314121 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:49.185729 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc625d8-c474-4c57-880d-51fb75314121 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:49.186148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc625d8-c474-4c57-880d-51fb75314121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:49.186281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5b27d-f640-4a9a-bf7a-3f32eba6d30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:49.187422 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5b27d-f640-4a9a-bf7a-3f32eba6d30d admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:49.187841 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5b27d-f640-4a9a-bf7a-3f32eba6d30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:49.188241 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5b27d-f640-4a9a-bf7a-3f32eba6d30d admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:49.191397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5b27d-f640-4a9a-bf7a-3f32eba6d30d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:49.200164 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5b27d-f640-4a9a-bf7a-3f32eba6d30d admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:49.235096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 822/1647] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:49.239908 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5b27d-f640-4a9a-bf7a-3f32eba6d30d admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:49.242209 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-cbc625d8-c474-4c57-880d-51fb75314121 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:49.243484 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cbc625d8-c474-4c57-880d-51fb75314121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:49.258906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:49.274307 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5b27d-f640-4a9a-bf7a-3f32eba6d30d admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:49.422428 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 826/1648] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:49.477953 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a105df5-ffaf-4fc2-97e7-8207881aeccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:49.479030 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a105df5-ffaf-4fc2-97e7-8207881aeccf admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:49.479313 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a105df5-ffaf-4fc2-97e7-8207881aeccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:49.479592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a105df5-ffaf-4fc2-97e7-8207881aeccf admin admin] PATH_INFO: `/v3/projects/b67255a6107044a18b4f70f4bd95e0e2/users/831a941b614f4ed7b3c9af6818b4c4cd/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:49.486471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:49.516481 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a105df5-ffaf-4fc2-97e7-8207881aeccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b67255a6107044a18b4f70f4bd95e0e2, user_id=831a941b614f4ed7b3c9af6818b4c4cd, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:49.519429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a105df5-ffaf-4fc2-97e7-8207881aeccf admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:49.556025 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 823/1649] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:49 2022] PUT /identity/v3/projects/b67255a6107044a18b4f70f4bd95e0e2/users/831a941b614f4ed7b3c9af6818b4c4cd/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:49.557701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc625d8-c474-4c57-880d-51fb75314121 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:49.572094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a105df5-ffaf-4fc2-97e7-8207881aeccf admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:49.574236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-2a105df5-ffaf-4fc2-97e7-8207881aeccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:49.604828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc625d8-c474-4c57-880d-51fb75314121 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:49.749660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc306b0-e7e9-42dc-a13e-51faad13afec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:49.749990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc306b0-e7e9-42dc-a13e-51faad13afec admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:49.750337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc306b0-e7e9-42dc-a13e-51faad13afec admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:49.750628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc306b0-e7e9-42dc-a13e-51faad13afec admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:49.751420 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc306b0-e7e9-42dc-a13e-51faad13afec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:49.754917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc306b0-e7e9-42dc-a13e-51faad13afec admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:49.776374 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a105df5-ffaf-4fc2-97e7-8207881aeccf admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:49.778591 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 827/1650] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:49.784964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc306b0-e7e9-42dc-a13e-51faad13afec admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:49.795083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81aab95-8a02-411f-b116-03d34fd8289e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:49.798717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aab95-8a02-411f-b116-03d34fd8289e service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:49.799197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aab95-8a02-411f-b116-03d34fd8289e service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:49.800082 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aab95-8a02-411f-b116-03d34fd8289e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:49.823676 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc306b0-e7e9-42dc-a13e-51faad13afec admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:49.844167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba11228-cfec-49be-b263-c5a4738ad524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:49.846132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba11228-cfec-49be-b263-c5a4738ad524 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:49.846434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba11228-cfec-49be-b263-c5a4738ad524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:49.846716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba11228-cfec-49be-b263-c5a4738ad524 admin admin] PATH_INFO: `/v3/projects/b67255a6107044a18b4f70f4bd95e0e2/users/831a941b614f4ed7b3c9af6818b4c4cd/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:49.847385 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81aab95-8a02-411f-b116-03d34fd8289e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:49.848887 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81aab95-8a02-411f-b116-03d34fd8289e service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:49.853185 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba11228-cfec-49be-b263-c5a4738ad524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b67255a6107044a18b4f70f4bd95e0e2, user_id=831a941b614f4ed7b3c9af6818b4c4cd, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:49.861491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba11228-cfec-49be-b263-c5a4738ad524 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:49.872697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 824/1651] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:23:49.898768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 828/1652] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:49 2022] PUT /identity/v3/projects/b67255a6107044a18b4f70f4bd95e0e2/users/831a941b614f4ed7b3c9af6818b4c4cd/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:51.757760 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81aab95-8a02-411f-b116-03d34fd8289e service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:51.759352 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-b81aab95-8a02-411f-b116-03d34fd8289e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:51.788130 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81aab95-8a02-411f-b116-03d34fd8289e service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:51.951696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1588d25-327a-4530-98c8-b2a8f027567b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:51.953435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1588d25-327a-4530-98c8-b2a8f027567b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:51.961184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1588d25-327a-4530-98c8-b2a8f027567b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:51.962283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1588d25-327a-4530-98c8-b2a8f027567b admin admin] PATH_INFO: `/v3/users/3fc7d5e916d54b61b0bd38c60634c52b` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:51.965945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1588d25-327a-4530-98c8-b2a8f027567b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fc7d5e916d54b61b0bd38c60634c52b)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:51.972765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1588d25-327a-4530-98c8-b2a8f027567b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:52.154781 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba11228-cfec-49be-b263-c5a4738ad524 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:52.156888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1ba11228-cfec-49be-b263-c5a4738ad524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:52.171212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-f1588d25-327a-4530-98c8-b2a8f027567b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fc7d5e916d54b61b0bd38c60634c52b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.192883 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba11228-cfec-49be-b263-c5a4738ad524 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:52.195568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-f1588d25-327a-4530-98c8-b2a8f027567b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fc7d5e916d54b61b0bd38c60634c52b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.208029 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602615c-10a6-4b62-8ab0-f458a80a285b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:52.208954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602615c-10a6-4b62-8ab0-f458a80a285b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.209357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602615c-10a6-4b62-8ab0-f458a80a285b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.209506 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602615c-10a6-4b62-8ab0-f458a80a285b admin admin] PATH_INFO: `/v3/projects/cb8ace75ec1e4e73bfc1f2c37328d733` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.213206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602615c-10a6-4b62-8ab0-f458a80a285b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb8ace75ec1e4e73bfc1f2c37328d733)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:52.215208 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602615c-10a6-4b62-8ab0-f458a80a285b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:52.248261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-f1588d25-327a-4530-98c8-b2a8f027567b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fc7d5e916d54b61b0bd38c60634c52b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.286419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-f1588d25-327a-4530-98c8-b2a8f027567b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fc7d5e916d54b61b0bd38c60634c52b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.289962 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-f1588d25-327a-4530-98c8-b2a8f027567b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fc7d5e916d54b61b0bd38c60634c52b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.313779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 825/1653] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:23:51 2022] DELETE /identity/v3/users/3fc7d5e916d54b61b0bd38c60634c52b => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:52.315511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccec2e-6ac4-46ce-9028-cdaee8cf2744 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.318770 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccec2e-6ac4-46ce-9028-cdaee8cf2744 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.320236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccec2e-6ac4-46ce-9028-cdaee8cf2744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.324519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6602615c-10a6-4b62-8ab0-f458a80a285b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb8ace75ec1e4e73bfc1f2c37328d733', 'request_id': 'req-6602615c-10a6-4b62-8ab0-f458a80a285b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.350411 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6602615c-10a6-4b62-8ab0-f458a80a285b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb8ace75ec1e4e73bfc1f2c37328d733', 'request_id': 'req-6602615c-10a6-4b62-8ab0-f458a80a285b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.352684 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:52.369547 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6602615c-10a6-4b62-8ab0-f458a80a285b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb8ace75ec1e4e73bfc1f2c37328d733', 'request_id': 'req-6602615c-10a6-4b62-8ab0-f458a80a285b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.385185 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6602615c-10a6-4b62-8ab0-f458a80a285b admin admin] The token cache is being invalidate because project cb8ace75ec1e4e73bfc1f2c37328d733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:23:52.385499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6602615c-10a6-4b62-8ab0-f458a80a285b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.388774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6602615c-10a6-4b62-8ab0-f458a80a285b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.393294 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 829/1654] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:23:52 2022] DELETE /identity/v3/projects/cb8ace75ec1e4e73bfc1f2c37328d733 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:52.393294 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e787b0-662e-4b93-8a4f-5cfc788750bc None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.393294 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e787b0-662e-4b93-8a4f-5cfc788750bc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.393294 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e787b0-662e-4b93-8a4f-5cfc788750bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.407827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-6fccec2e-6ac4-46ce-9028-cdaee8cf2744 None None] MFA Rules not processed for user `cb06eeba738b452fab3986de7e1f7856`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:52.417789 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6fccec2e-6ac4-46ce-9028-cdaee8cf2744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:52.442426 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f1e787b0-662e-4b93-8a4f-5cfc788750bc None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:52.443840 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f1e787b0-662e-4b93-8a4f-5cfc788750bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:52.471461 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:52.607027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 830/1655] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:52.607989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e787b0-662e-4b93-8a4f-5cfc788750bc None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:52.610488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 826/1656] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:52.617686 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f1e787b0-662e-4b93-8a4f-5cfc788750bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:52.622130 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb3aa7-57b2-4945-88d6-2273b405f9d0 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.622421 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb3aa7-57b2-4945-88d6-2273b405f9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.625602 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb3aa7-57b2-4945-88d6-2273b405f9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.642311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e787b0-662e-4b93-8a4f-5cfc788750bc None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:52.656726 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e0867-d259-4039-9eff-063d76c1d61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:52.657635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e0867-d259-4039-9eff-063d76c1d61f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.657922 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e0867-d259-4039-9eff-063d76c1d61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.658201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e0867-d259-4039-9eff-063d76c1d61f admin admin] PATH_INFO: `/v3/users/31cca6ef8541472a8a10eae2b345b6b2` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.659517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e0867-d259-4039-9eff-063d76c1d61f admin admin] RBAC: Authorizing `identity:delete_user(user_id=31cca6ef8541472a8a10eae2b345b6b2)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:52.662083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e0867-d259-4039-9eff-063d76c1d61f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:52.685576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-2ceb3aa7-57b2-4945-88d6-2273b405f9d0 None None] MFA Rules not processed for user `cb06eeba738b452fab3986de7e1f7856`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:52.687089 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-2ceb3aa7-57b2-4945-88d6-2273b405f9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:52.713856 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 827/1657] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:52.713856 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb3aa7-57b2-4945-88d6-2273b405f9d0 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:52.724092 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-2ceb3aa7-57b2-4945-88d6-2273b405f9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:52.753589 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-aa7e0867-d259-4039-9eff-063d76c1d61f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31cca6ef8541472a8a10eae2b345b6b2'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.773157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-aa7e0867-d259-4039-9eff-063d76c1d61f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31cca6ef8541472a8a10eae2b345b6b2'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.773620 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-aa7e0867-d259-4039-9eff-063d76c1d61f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31cca6ef8541472a8a10eae2b345b6b2'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.785219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-aa7e0867-d259-4039-9eff-063d76c1d61f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31cca6ef8541472a8a10eae2b345b6b2'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.824819 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-aa7e0867-d259-4039-9eff-063d76c1d61f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31cca6ef8541472a8a10eae2b345b6b2'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.862304 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 831/1658] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:23:52 2022] DELETE /identity/v3/users/31cca6ef8541472a8a10eae2b345b6b2 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:52.863736 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df13e4-ea79-4851-9263-2c371438aee8 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.864032 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df13e4-ea79-4851-9263-2c371438aee8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.864383 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df13e4-ea79-4851-9263-2c371438aee8 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.865980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 832/1659] 10.222.0.43 () {54 vars in 962 bytes} [Wed Jul 20 19:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:23:52.867251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f79621-549a-480c-a7a6-6d53c5946fc0 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.867542 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f79621-549a-480c-a7a6-6d53c5946fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.867824 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f79621-549a-480c-a7a6-6d53c5946fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.887005 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb3aa7-57b2-4945-88d6-2273b405f9d0 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:52.927984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-77f79621-549a-480c-a7a6-6d53c5946fc0 None None] MFA Rules not processed for user `831a941b614f4ed7b3c9af6818b4c4cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:52.929264 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-77f79621-549a-480c-a7a6-6d53c5946fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:52.966152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331c80e-2589-4687-8201-ac0f9d7ff757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:52.967070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331c80e-2589-4687-8201-ac0f9d7ff757 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.967182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331c80e-2589-4687-8201-ac0f9d7ff757 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.968140 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331c80e-2589-4687-8201-ac0f9d7ff757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.969290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f331c80e-2589-4687-8201-ac0f9d7ff757 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:53.060251 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:53.064703 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331c80e-2589-4687-8201-ac0f9d7ff757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:53.068680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331c80e-2589-4687-8201-ac0f9d7ff757 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:53.085662 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 833/1660] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:52 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:53.087014 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499114a1-7c1f-41e8-b612-8c47504e7264 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:53.087308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499114a1-7c1f-41e8-b612-8c47504e7264 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:53.087587 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499114a1-7c1f-41e8-b612-8c47504e7264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:53.091185 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 828/1661] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:53.149697 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:53.173163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-499114a1-7c1f-41e8-b612-8c47504e7264 None None] MFA Rules not processed for user `267b178236e046e4bc694b2da6f86eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:53.174689 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-499114a1-7c1f-41e8-b612-8c47504e7264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:53.338775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 834/1662] 10.222.0.43 () {58 vars in 1059 bytes} [Wed Jul 20 19:23:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:53.354461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331c80e-2589-4687-8201-ac0f9d7ff757 service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:53.379606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331c80e-2589-4687-8201-ac0f9d7ff757 service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:53.387689 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f153d4a-ee09-4664-9c0e-beb93b03334b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:53.388440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f153d4a-ee09-4664-9c0e-beb93b03334b service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:53.388901 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f153d4a-ee09-4664-9c0e-beb93b03334b service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:53.389188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f153d4a-ee09-4664-9c0e-beb93b03334b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:53.391211 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-8f153d4a-ee09-4664-9c0e-beb93b03334b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:53.495651 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f153d4a-ee09-4664-9c0e-beb93b03334b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:53.502174 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f153d4a-ee09-4664-9c0e-beb93b03334b service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:53.549651 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 829/1663] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:54.374281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46521ec-73ce-4a81-bf60-089d01a78d2d None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:54.374907 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46521ec-73ce-4a81-bf60-089d01a78d2d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:54.374907 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46521ec-73ce-4a81-bf60-089d01a78d2d None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:54.376967 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 835/1664] 10.222.0.43 () {54 vars in 912 bytes} [Wed Jul 20 19:23:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:23:54.397839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e100677-d3a4-427c-812d-0caf5f3811f7 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:54.398396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e100677-d3a4-427c-812d-0caf5f3811f7 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:54.398396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e100677-d3a4-427c-812d-0caf5f3811f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:54.439923 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-3e100677-d3a4-427c-812d-0caf5f3811f7 None None] MFA Rules not processed for user `267b178236e046e4bc694b2da6f86eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:54.445074 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-3e100677-d3a4-427c-812d-0caf5f3811f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:54.476127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 830/1665] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jul 20 19:23:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:54.494023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46521ec-73ce-4a81-bf60-089d01a78d2d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:54.495842 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a46521ec-73ce-4a81-bf60-089d01a78d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:54.770843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46521ec-73ce-4a81-bf60-089d01a78d2d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:54.782234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236a379-6492-4682-9db7-bb48814915ec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3e28557896e41c893f0de2590bc96d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:54.784924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236a379-6492-4682-9db7-bb48814915ec service glance] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:54.786789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236a379-6492-4682-9db7-bb48814915ec service glance] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:54.787075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236a379-6492-4682-9db7-bb48814915ec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:54.822044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236a379-6492-4682-9db7-bb48814915ec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:54.825877 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236a379-6492-4682-9db7-bb48814915ec service glance] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:54.863334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 836/1666] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:55.295898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e100677-d3a4-427c-812d-0caf5f3811f7 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:55.298562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-3e100677-d3a4-427c-812d-0caf5f3811f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:55.327945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e100677-d3a4-427c-812d-0caf5f3811f7 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:55.350781 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c36e3-5992-46fc-9767-3809e4e69014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:55.350781 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c36e3-5992-46fc-9767-3809e4e69014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:55.350781 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c36e3-5992-46fc-9767-3809e4e69014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:55.351166 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c36e3-5992-46fc-9767-3809e4e69014 admin admin] PATH_INFO: `/v3/projects/4e6e5965bb1e4d8c8577134ce542a311` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:55.353114 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c36e3-5992-46fc-9767-3809e4e69014 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e6e5965bb1e4d8c8577134ce542a311)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:55.357082 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: WARNING py.warnings [None req-2c5c36e3-5992-46fc-9767-3809e4e69014 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:55.357082 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: warnings.warn(msg) Jul 20 19:23:55.357082 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: Jul 20 19:23:55.361250 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c36e3-5992-46fc-9767-3809e4e69014 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:55.570862 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-2c5c36e3-5992-46fc-9767-3809e4e69014 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e6e5965bb1e4d8c8577134ce542a311', 'request_id': 'req-2c5c36e3-5992-46fc-9767-3809e4e69014'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:55.603366 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-2c5c36e3-5992-46fc-9767-3809e4e69014 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e6e5965bb1e4d8c8577134ce542a311', 'request_id': 'req-2c5c36e3-5992-46fc-9767-3809e4e69014'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:55.624398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-2c5c36e3-5992-46fc-9767-3809e4e69014 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e6e5965bb1e4d8c8577134ce542a311', 'request_id': 'req-2c5c36e3-5992-46fc-9767-3809e4e69014'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:55.669186 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-2c5c36e3-5992-46fc-9767-3809e4e69014 admin admin] The token cache is being invalidate because project 4e6e5965bb1e4d8c8577134ce542a311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:23:55.670013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-2c5c36e3-5992-46fc-9767-3809e4e69014 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:55.675810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-2c5c36e3-5992-46fc-9767-3809e4e69014 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:55.680035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 831/1667] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:23:55 2022] DELETE /identity/v3/projects/4e6e5965bb1e4d8c8577134ce542a311 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:55.684115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236a379-6492-4682-9db7-bb48814915ec service glance] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:55.706292 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c36e3-5992-46fc-9767-3809e4e69014 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:55.713667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-2c5c36e3-5992-46fc-9767-3809e4e69014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:55.727650 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236a379-6492-4682-9db7-bb48814915ec service glance] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:55.889971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dddee4-c718-4c6d-81a3-50ced0c91d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:55.891245 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dddee4-c718-4c6d-81a3-50ced0c91d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:55.891704 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dddee4-c718-4c6d-81a3-50ced0c91d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:55.892243 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dddee4-c718-4c6d-81a3-50ced0c91d50 admin admin] PATH_INFO: `/v3/users/4d8cc7f472844daea4cfb6afa6d8db3e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:55.894367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c36e3-5992-46fc-9767-3809e4e69014 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:55.895232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dddee4-c718-4c6d-81a3-50ced0c91d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d8cc7f472844daea4cfb6afa6d8db3e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:55.899762 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dddee4-c718-4c6d-81a3-50ced0c91d50 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:55.907534 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66c1d0-837b-4a2d-accd-321c51eb2009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:55.908647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66c1d0-837b-4a2d-accd-321c51eb2009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:55.909275 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66c1d0-837b-4a2d-accd-321c51eb2009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:55.910698 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66c1d0-837b-4a2d-accd-321c51eb2009 admin admin] PATH_INFO: `/v3/projects/39dd51065ee3444fb808ac75eac7973a` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:55.913194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66c1d0-837b-4a2d-accd-321c51eb2009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39dd51065ee3444fb808ac75eac7973a)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:55.921471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66c1d0-837b-4a2d-accd-321c51eb2009 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:55.953199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:55.953681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:56.030481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-50dddee4-c718-4c6d-81a3-50ced0c91d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d8cc7f472844daea4cfb6afa6d8db3e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:56.060948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-50dddee4-c718-4c6d-81a3-50ced0c91d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d8cc7f472844daea4cfb6afa6d8db3e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:56.064875 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-50dddee4-c718-4c6d-81a3-50ced0c91d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d8cc7f472844daea4cfb6afa6d8db3e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:56.085941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-fc66c1d0-837b-4a2d-accd-321c51eb2009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39dd51065ee3444fb808ac75eac7973a', 'request_id': 'req-fc66c1d0-837b-4a2d-accd-321c51eb2009'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:56.099669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-fc66c1d0-837b-4a2d-accd-321c51eb2009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39dd51065ee3444fb808ac75eac7973a', 'request_id': 'req-fc66c1d0-837b-4a2d-accd-321c51eb2009'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:56.106088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-50dddee4-c718-4c6d-81a3-50ced0c91d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d8cc7f472844daea4cfb6afa6d8db3e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:56.128751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-fc66c1d0-837b-4a2d-accd-321c51eb2009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39dd51065ee3444fb808ac75eac7973a', 'request_id': 'req-fc66c1d0-837b-4a2d-accd-321c51eb2009'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:56.145695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-50dddee4-c718-4c6d-81a3-50ced0c91d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d8cc7f472844daea4cfb6afa6d8db3e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:56.156993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-fc66c1d0-837b-4a2d-accd-321c51eb2009 admin admin] The token cache is being invalidate because project 39dd51065ee3444fb808ac75eac7973a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:23:56.157619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-fc66c1d0-837b-4a2d-accd-321c51eb2009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:56.159769 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-fc66c1d0-837b-4a2d-accd-321c51eb2009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:56.166087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 837/1668] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:23:55 2022] DELETE /identity/v3/users/4d8cc7f472844daea4cfb6afa6d8db3e => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:56.166532 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 832/1669] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:23:55 2022] DELETE /identity/v3/projects/39dd51065ee3444fb808ac75eac7973a => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:56.172417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be91563-9eea-4906-b610-7ca02cc82d8f None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:56.172723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be91563-9eea-4906-b610-7ca02cc82d8f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:56.173004 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be91563-9eea-4906-b610-7ca02cc82d8f None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:56.173652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbb4f6-d036-4cf6-819c-9384f5e968cf None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:56.174018 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbb4f6-d036-4cf6-819c-9384f5e968cf None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:56.175396 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 838/1670] 10.222.0.43 () {54 vars in 912 bytes} [Wed Jul 20 19:23:56 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:23:56.175522 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbb4f6-d036-4cf6-819c-9384f5e968cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:56.181617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83f32d-3f3c-4bd6-8027-40659b0c94a7 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:56.181926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83f32d-3f3c-4bd6-8027-40659b0c94a7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:56.182178 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83f32d-3f3c-4bd6-8027-40659b0c94a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:56.230347 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-68bbb4f6-d036-4cf6-819c-9384f5e968cf None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:56.237777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-68bbb4f6-d036-4cf6-819c-9384f5e968cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:56.273258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ed83f32d-3f3c-4bd6-8027-40659b0c94a7 None None] MFA Rules not processed for user `74327a10eaab43e0af02050da1a24f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:56.276427 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ed83f32d-3f3c-4bd6-8027-40659b0c94a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:56.404528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 839/1671] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jul 20 19:23:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:23:56.425003 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83f32d-3f3c-4bd6-8027-40659b0c94a7 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:56.427404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ed83f32d-3f3c-4bd6-8027-40659b0c94a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:56.453968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83f32d-3f3c-4bd6-8027-40659b0c94a7 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:56.469909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 833/1672] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:56.476561 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182a414-8f7b-4390-8cec-8010289675b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:56.477442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182a414-8f7b-4390-8cec-8010289675b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:56.477740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182a414-8f7b-4390-8cec-8010289675b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:56.478122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182a414-8f7b-4390-8cec-8010289675b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:56.478337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027fad0-025a-4bc5-ab5a-85dbd37489a9 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:56.481693 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027fad0-025a-4bc5-ab5a-85dbd37489a9 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:56.482026 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027fad0-025a-4bc5-ab5a-85dbd37489a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:56.505655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182a414-8f7b-4390-8cec-8010289675b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:56.509168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182a414-8f7b-4390-8cec-8010289675b0 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:56.544278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 840/1673] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:56.551955 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147c3cd-9da0-4e09-8b4f-7b88ff9ac6f6 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:56.553149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-c027fad0-025a-4bc5-ab5a-85dbd37489a9 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:56.554408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147c3cd-9da0-4e09-8b4f-7b88ff9ac6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:56.555386 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147c3cd-9da0-4e09-8b4f-7b88ff9ac6f6 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:56.556266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c027fad0-025a-4bc5-ab5a-85dbd37489a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:56.558427 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 841/1674] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jul 20 19:23:56 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:23:56.584156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 834/1675] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:56.588119 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147c3cd-9da0-4e09-8b4f-7b88ff9ac6f6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:56.628241 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147c3cd-9da0-4e09-8b4f-7b88ff9ac6f6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:56.666597 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd091c22-a8f0-4a98-ae7e-847509a0fda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:56.667298 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd091c22-a8f0-4a98-ae7e-847509a0fda4 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:56.668775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd091c22-a8f0-4a98-ae7e-847509a0fda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:56.668775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd091c22-a8f0-4a98-ae7e-847509a0fda4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:56.668775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd091c22-a8f0-4a98-ae7e-847509a0fda4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:56.669133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd091c22-a8f0-4a98-ae7e-847509a0fda4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:56.670574 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd091c22-a8f0-4a98-ae7e-847509a0fda4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:56.712981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48bbfa-a4ac-48ac-bd2e-687237976e12 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:56.713296 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48bbfa-a4ac-48ac-bd2e-687237976e12 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:56.713577 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48bbfa-a4ac-48ac-bd2e-687237976e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:56.718429 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 842/1676] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:56.726871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd091c22-a8f0-4a98-ae7e-847509a0fda4 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:56.749515 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-af48bbfa-a4ac-48ac-bd2e-687237976e12 None None] MFA Rules not processed for user `ae581d3dda354e07894cb5205d1b2c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:56.750826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-af48bbfa-a4ac-48ac-bd2e-687237976e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:56.756386 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd091c22-a8f0-4a98-ae7e-847509a0fda4 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:56.781277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5018d-ee39-49c1-9889-b2e2297ff240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:56.781986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5018d-ee39-49c1-9889-b2e2297ff240 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:56.782275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5018d-ee39-49c1-9889-b2e2297ff240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:56.782556 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5018d-ee39-49c1-9889-b2e2297ff240 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:56.783283 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5018d-ee39-49c1-9889-b2e2297ff240 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:56.785310 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5018d-ee39-49c1-9889-b2e2297ff240 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:56.850393 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 843/1677] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:56 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:56.853179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5018d-ee39-49c1-9889-b2e2297ff240 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:56.924551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 835/1678] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jul 20 19:23:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:23:56.953282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:56.977378 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5018d-ee39-49c1-9889-b2e2297ff240 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:56.984571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ea454-e3d9-4ffd-9ff4-2eca9d67ff36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:56.985523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ea454-e3d9-4ffd-9ff4-2eca9d67ff36 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:56.985951 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ea454-e3d9-4ffd-9ff4-2eca9d67ff36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:56.986551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ea454-e3d9-4ffd-9ff4-2eca9d67ff36 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:56.987416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ea454-e3d9-4ffd-9ff4-2eca9d67ff36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:56.990481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ea454-e3d9-4ffd-9ff4-2eca9d67ff36 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:57.023021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 844/1679] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:56 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:57.023751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48bbfa-a4ac-48ac-bd2e-687237976e12 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:57.050944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48bbfa-a4ac-48ac-bd2e-687237976e12 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:57.062831 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040503ac-9061-4f3f-bf8f-08227a820a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:57.063548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040503ac-9061-4f3f-bf8f-08227a820a2c admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:57.063839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040503ac-9061-4f3f-bf8f-08227a820a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:57.064124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040503ac-9061-4f3f-bf8f-08227a820a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:57.067942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040503ac-9061-4f3f-bf8f-08227a820a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:57.069769 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040503ac-9061-4f3f-bf8f-08227a820a2c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:57.094339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b9491-0062-4fa5-ae2e-94c396d73de5 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:57.095139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 836/1680] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:23:57.101609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040503ac-9061-4f3f-bf8f-08227a820a2c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:57.102615 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b9491-0062-4fa5-ae2e-94c396d73de5 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:57.103159 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b9491-0062-4fa5-ae2e-94c396d73de5 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:57.104932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 845/1681] 10.222.0.43 () {54 vars in 961 bytes} [Wed Jul 20 19:23:57 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:23:57.111659 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43fa3de-9f46-4d26-8fc0-29dbb4f72b85 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:57.112992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43fa3de-9f46-4d26-8fc0-29dbb4f72b85 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:57.113427 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43fa3de-9f46-4d26-8fc0-29dbb4f72b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:57.117355 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040503ac-9061-4f3f-bf8f-08227a820a2c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:57.130011 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3d282-955e-4b20-bc25-a04e1cf50339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:57.131447 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3d282-955e-4b20-bc25-a04e1cf50339 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:57.132018 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3d282-955e-4b20-bc25-a04e1cf50339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:57.132482 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3d282-955e-4b20-bc25-a04e1cf50339 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:57.133567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3d282-955e-4b20-bc25-a04e1cf50339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:57.135892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3d282-955e-4b20-bc25-a04e1cf50339 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:57.150865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 837/1682] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:57.155294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3d282-955e-4b20-bc25-a04e1cf50339 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:57.175222 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3d282-955e-4b20-bc25-a04e1cf50339 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:57.184586 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272375ca-2034-429b-957c-12d628f99f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:57.184822 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e43fa3de-9f46-4d26-8fc0-29dbb4f72b85 None None] MFA Rules not processed for user `ae581d3dda354e07894cb5205d1b2c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:57.185393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272375ca-2034-429b-957c-12d628f99f7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:57.185686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272375ca-2034-429b-957c-12d628f99f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:57.185783 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e43fa3de-9f46-4d26-8fc0-29dbb4f72b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:57.185987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272375ca-2034-429b-957c-12d628f99f7a admin admin] PATH_INFO: `/v3/projects/1e82de68ad9c430ba48b04b8e366c746/users/8b86b9f69ddd4d88b239866662407bcc/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:57.195574 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:57.215377 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272375ca-2034-429b-957c-12d628f99f7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e82de68ad9c430ba48b04b8e366c746, user_id=8b86b9f69ddd4d88b239866662407bcc, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:57.220522 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272375ca-2034-429b-957c-12d628f99f7a admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:57.263206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 838/1683] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:57 2022] PUT /identity/v3/projects/1e82de68ad9c430ba48b04b8e366c746/users/8b86b9f69ddd4d88b239866662407bcc/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:57.275481 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee7ff8-76da-4029-8257-360d56f04a51 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:57.275781 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee7ff8-76da-4029-8257-360d56f04a51 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:57.276127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee7ff8-76da-4029-8257-360d56f04a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:57.338909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-dbee7ff8-76da-4029-8257-360d56f04a51 None None] MFA Rules not processed for user `8b86b9f69ddd4d88b239866662407bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:57.340419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-dbee7ff8-76da-4029-8257-360d56f04a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:57.401486 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 846/1684] 10.222.0.43 () {58 vars in 1058 bytes} [Wed Jul 20 19:23:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:23:57.429760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43fa3de-9f46-4d26-8fc0-29dbb4f72b85 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:57.448417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43fa3de-9f46-4d26-8fc0-29dbb4f72b85 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:57.513061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 839/1685] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:57.522495 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee7ff8-76da-4029-8257-360d56f04a51 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:57.683860 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5914e-34ee-4deb-8dd8-eb78149a73a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae581d3dda354e07894cb5205d1b2c5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:57.684587 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5914e-34ee-4deb-8dd8-eb78149a73a0 service placement] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:57.684873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5914e-34ee-4deb-8dd8-eb78149a73a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:57.685148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5914e-34ee-4deb-8dd8-eb78149a73a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:57.724221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5914e-34ee-4deb-8dd8-eb78149a73a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:57.730034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5914e-34ee-4deb-8dd8-eb78149a73a0 service placement] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:57.735063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee7ff8-76da-4029-8257-360d56f04a51 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:57.756490 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ebf4e-9cba-42b7-ac2c-9cdfb7503f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:57.757064 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ebf4e-9cba-42b7-ac2c-9cdfb7503f31 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:57.757366 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ebf4e-9cba-42b7-ac2c-9cdfb7503f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:57.757648 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ebf4e-9cba-42b7-ac2c-9cdfb7503f31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:57.758554 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ebf4e-9cba-42b7-ac2c-9cdfb7503f31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:57.760449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ebf4e-9cba-42b7-ac2c-9cdfb7503f31 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:57.762622 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 847/1686] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:57.830799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 840/1687] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:57 2022] POST /identity/v3/projects => generated 377 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:57.831382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5914e-34ee-4deb-8dd8-eb78149a73a0 service placement] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:57.845742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5914e-34ee-4deb-8dd8-eb78149a73a0 service placement] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:57.962330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5009b8-8ffb-4e76-a5a0-405842cdc91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:57.963194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5009b8-8ffb-4e76-a5a0-405842cdc91f admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:57.963918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5009b8-8ffb-4e76-a5a0-405842cdc91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:57.964833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5009b8-8ffb-4e76-a5a0-405842cdc91f admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:57.967001 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5009b8-8ffb-4e76-a5a0-405842cdc91f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:57.976590 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5009b8-8ffb-4e76-a5a0-405842cdc91f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:57.990000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:58.017644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 848/1688] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:57 2022] POST /identity/v3/users => generated 295 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:58.021617 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ebf4e-9cba-42b7-ac2c-9cdfb7503f31 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:58.038893 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ebf4e-9cba-42b7-ac2c-9cdfb7503f31 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:58.055741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76227798-b51c-4105-bb96-82111491d5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:58.056979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76227798-b51c-4105-bb96-82111491d5af admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:58.057274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76227798-b51c-4105-bb96-82111491d5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:58.057610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76227798-b51c-4105-bb96-82111491d5af admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:58.058375 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76227798-b51c-4105-bb96-82111491d5af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:58.059996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76227798-b51c-4105-bb96-82111491d5af admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:58.095927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 841/1689] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:58.097730 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5009b8-8ffb-4e76-a5a0-405842cdc91f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:58.114812 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5009b8-8ffb-4e76-a5a0-405842cdc91f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:58.125664 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46097962-d483-4f91-8ad3-b1832167b438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:58.127305 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46097962-d483-4f91-8ad3-b1832167b438 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:58.128576 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46097962-d483-4f91-8ad3-b1832167b438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:58.130266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46097962-d483-4f91-8ad3-b1832167b438 admin admin] PATH_INFO: `/v3/projects/e4f3accc405047b19af7e3a5a4d23261/users/4bc38af54a4c4ec3a9041d1d624b987a/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:58.152392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46097962-d483-4f91-8ad3-b1832167b438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4f3accc405047b19af7e3a5a4d23261, user_id=4bc38af54a4c4ec3a9041d1d624b987a, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:58.155368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46097962-d483-4f91-8ad3-b1832167b438 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:58.187998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 849/1690] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:58 2022] PUT /identity/v3/projects/e4f3accc405047b19af7e3a5a4d23261/users/4bc38af54a4c4ec3a9041d1d624b987a/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:58.193971 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76227798-b51c-4105-bb96-82111491d5af admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:58.213859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76227798-b51c-4105-bb96-82111491d5af admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:58.335884 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:58.406291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fdd09-a612-428c-b7ca-e2f986f26587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:58.407978 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fdd09-a612-428c-b7ca-e2f986f26587 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:58.413455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fdd09-a612-428c-b7ca-e2f986f26587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:58.414627 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fdd09-a612-428c-b7ca-e2f986f26587 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:58.417217 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fdd09-a612-428c-b7ca-e2f986f26587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:58.424676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fdd09-a612-428c-b7ca-e2f986f26587 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:58.468442 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 842/1691] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:58.472799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46097962-d483-4f91-8ad3-b1832167b438 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:58.504356 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46097962-d483-4f91-8ad3-b1832167b438 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:58.519976 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdbb0f-7d33-483f-84d9-bac889502385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:58.522568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdbb0f-7d33-483f-84d9-bac889502385 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:58.522863 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdbb0f-7d33-483f-84d9-bac889502385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:58.523310 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdbb0f-7d33-483f-84d9-bac889502385 admin admin] PATH_INFO: `/v3/projects/e4f3accc405047b19af7e3a5a4d23261/users/4bc38af54a4c4ec3a9041d1d624b987a/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:58.532470 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdbb0f-7d33-483f-84d9-bac889502385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4f3accc405047b19af7e3a5a4d23261, user_id=4bc38af54a4c4ec3a9041d1d624b987a, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:58.537076 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdbb0f-7d33-483f-84d9-bac889502385 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:58.579607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 850/1692] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:58 2022] PUT /identity/v3/projects/e4f3accc405047b19af7e3a5a4d23261/users/4bc38af54a4c4ec3a9041d1d624b987a/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:58.591502 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e0810-f7d8-42a6-b666-48adb2dab088 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:58.592035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e0810-f7d8-42a6-b666-48adb2dab088 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:58.592612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e0810-f7d8-42a6-b666-48adb2dab088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:58.664869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-be9e0810-f7d8-42a6-b666-48adb2dab088 None None] MFA Rules not processed for user `4bc38af54a4c4ec3a9041d1d624b987a`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:58.668635 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-be9e0810-f7d8-42a6-b666-48adb2dab088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:58.959170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 843/1693] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:58.971143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdbb0f-7d33-483f-84d9-bac889502385 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:58.973012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6ffdbb0f-7d33-483f-84d9-bac889502385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:59.190006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdbb0f-7d33-483f-84d9-bac889502385 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:59.197519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c88856-f456-4106-97ed-44a7062ab356 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:59.198504 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c88856-f456-4106-97ed-44a7062ab356 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:59.198504 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c88856-f456-4106-97ed-44a7062ab356 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:59.198792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c88856-f456-4106-97ed-44a7062ab356 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:59.296889 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e0810-f7d8-42a6-b666-48adb2dab088 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:59.299541 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-be9e0810-f7d8-42a6-b666-48adb2dab088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:59.351417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c88856-f456-4106-97ed-44a7062ab356 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:59.353971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c88856-f456-4106-97ed-44a7062ab356 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:59.371654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 851/1694] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:59.383678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c88856-f456-4106-97ed-44a7062ab356 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:59.523713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e0810-f7d8-42a6-b666-48adb2dab088 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:59.537980 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788309c9-68d0-42d4-8f70-d5bb8b0fa246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:59.538407 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788309c9-68d0-42d4-8f70-d5bb8b0fa246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:59.538696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788309c9-68d0-42d4-8f70-d5bb8b0fa246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:59.540533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788309c9-68d0-42d4-8f70-d5bb8b0fa246 admin admin] PATH_INFO: `/v3/projects/e6a6f6eb1aa84ffab9ec5117df9e0f5b` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:59.547936 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788309c9-68d0-42d4-8f70-d5bb8b0fa246 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6a6f6eb1aa84ffab9ec5117df9e0f5b)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:59.552930 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788309c9-68d0-42d4-8f70-d5bb8b0fa246 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:59.582103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c88856-f456-4106-97ed-44a7062ab356 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:59.590412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bdeac-b235-4645-b92e-d2f6da6a74c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae581d3dda354e07894cb5205d1b2c5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:59.591244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bdeac-b235-4645-b92e-d2f6da6a74c7 service placement] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:59.591599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bdeac-b235-4645-b92e-d2f6da6a74c7 service placement] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:59.591948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bdeac-b235-4645-b92e-d2f6da6a74c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:59.607946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b53bdeac-b235-4645-b92e-d2f6da6a74c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:59.673192 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:59.680869 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bdeac-b235-4645-b92e-d2f6da6a74c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:59.684471 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bdeac-b235-4645-b92e-d2f6da6a74c7 service placement] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:59.724161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 852/1695] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:23:59.749989 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-788309c9-68d0-42d4-8f70-d5bb8b0fa246 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6a6f6eb1aa84ffab9ec5117df9e0f5b', 'request_id': 'req-788309c9-68d0-42d4-8f70-d5bb8b0fa246'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:59.771495 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-788309c9-68d0-42d4-8f70-d5bb8b0fa246 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6a6f6eb1aa84ffab9ec5117df9e0f5b', 'request_id': 'req-788309c9-68d0-42d4-8f70-d5bb8b0fa246'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:59.791419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-788309c9-68d0-42d4-8f70-d5bb8b0fa246 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6a6f6eb1aa84ffab9ec5117df9e0f5b', 'request_id': 'req-788309c9-68d0-42d4-8f70-d5bb8b0fa246'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:59.812064 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:59.820532 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-788309c9-68d0-42d4-8f70-d5bb8b0fa246 admin admin] The token cache is being invalidate because project e6a6f6eb1aa84ffab9ec5117df9e0f5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:23:59.823062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-788309c9-68d0-42d4-8f70-d5bb8b0fa246 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:59.824615 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-788309c9-68d0-42d4-8f70-d5bb8b0fa246 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:59.834175 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 844/1696] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:23:59 2022] DELETE /identity/v3/projects/e6a6f6eb1aa84ffab9ec5117df9e0f5b => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:59.836428 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dda0c-2411-4e35-a936-461268ad8dcf None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:59.836728 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dda0c-2411-4e35-a936-461268ad8dcf None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:59.837011 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dda0c-2411-4e35-a936-461268ad8dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:59.891254 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-580dda0c-2411-4e35-a936-461268ad8dcf None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:59.895612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-580dda0c-2411-4e35-a936-461268ad8dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:00.222617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 853/1697] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:00.231772 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05e0e3-1e61-4c44-a787-ce0919b0b931 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:00.235058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05e0e3-1e61-4c44-a787-ce0919b0b931 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:00.235715 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05e0e3-1e61-4c44-a787-ce0919b0b931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:00.268165 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12895157-f1b3-4b5d-ae15-cdd66f0b59b6 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:00.268753 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12895157-f1b3-4b5d-ae15-cdd66f0b59b6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:00.269306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12895157-f1b3-4b5d-ae15-cdd66f0b59b6 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:00.273230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 854/1698] 10.222.0.43 () {54 vars in 962 bytes} [Wed Jul 20 19:24:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:24:00.296127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9966b9a-0cd6-4f19-a547-30c16968bba7 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:00.296918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9966b9a-0cd6-4f19-a547-30c16968bba7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:00.297250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9966b9a-0cd6-4f19-a547-30c16968bba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:00.321601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-5c05e0e3-1e61-4c44-a787-ce0919b0b931 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:00.329404 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5c05e0e3-1e61-4c44-a787-ce0919b0b931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:00.368314 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e9966b9a-0cd6-4f19-a547-30c16968bba7 None None] MFA Rules not processed for user `267b178236e046e4bc694b2da6f86eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:00.371584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e9966b9a-0cd6-4f19-a547-30c16968bba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:00.382082 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 845/1699] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:00.389338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05e0e3-1e61-4c44-a787-ce0919b0b931 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:00.463291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05e0e3-1e61-4c44-a787-ce0919b0b931 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:00.499548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0092126-8128-4322-ba61-b1025fe58aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:00.500247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0092126-8128-4322-ba61-b1025fe58aea admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:00.501496 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0092126-8128-4322-ba61-b1025fe58aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:00.501496 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0092126-8128-4322-ba61-b1025fe58aea admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:00.514274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0092126-8128-4322-ba61-b1025fe58aea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:00.516397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0092126-8128-4322-ba61-b1025fe58aea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:00.530123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0092126-8128-4322-ba61-b1025fe58aea admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:00.546186 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 855/1700] 10.222.0.43 () {58 vars in 1059 bytes} [Wed Jul 20 19:24:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:00.553880 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36845add-802e-4378-ac61-15486ce9fc1b None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:00.554172 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36845add-802e-4378-ac61-15486ce9fc1b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:00.554452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36845add-802e-4378-ac61-15486ce9fc1b None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:00.555841 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 856/1701] 10.222.0.43 () {54 vars in 912 bytes} [Wed Jul 20 19:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:24:00.565837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb0a2c-492a-404e-aefb-1fef25f1528e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:00.566125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb0a2c-492a-404e-aefb-1fef25f1528e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:00.566406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb0a2c-492a-404e-aefb-1fef25f1528e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:00.607774 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 846/1702] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:00.608852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0092126-8128-4322-ba61-b1025fe58aea admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:00.623797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-72fb0a2c-492a-404e-aefb-1fef25f1528e None None] MFA Rules not processed for user `267b178236e046e4bc694b2da6f86eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:00.625401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-72fb0a2c-492a-404e-aefb-1fef25f1528e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:00.646933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0092126-8128-4322-ba61-b1025fe58aea admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:00.648316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 857/1703] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jul 20 19:24:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:00.662851 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798d003-f41d-447a-bf23-27afbbb7a484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:00.668268 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798d003-f41d-447a-bf23-27afbbb7a484 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:00.668592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798d003-f41d-447a-bf23-27afbbb7a484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:00.669273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798d003-f41d-447a-bf23-27afbbb7a484 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:00.669994 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e798d003-f41d-447a-bf23-27afbbb7a484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:00.675035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e798d003-f41d-447a-bf23-27afbbb7a484 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:00.712278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb0a2c-492a-404e-aefb-1fef25f1528e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:00.714727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-72fb0a2c-492a-404e-aefb-1fef25f1528e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:00.757899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 847/1704] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:00 2022] POST /identity/v3/projects => generated 367 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:00.762863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798d003-f41d-447a-bf23-27afbbb7a484 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:00.897772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb0a2c-492a-404e-aefb-1fef25f1528e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:00.908643 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798d003-f41d-447a-bf23-27afbbb7a484 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:00.915120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b9bae-e436-4cae-ac7d-5e6b8c7a67a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:00.915939 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b9bae-e436-4cae-ac7d-5e6b8c7a67a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:00.916106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b9bae-e436-4cae-ac7d-5e6b8c7a67a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:00.916561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b9bae-e436-4cae-ac7d-5e6b8c7a67a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:00.917435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b9bae-e436-4cae-ac7d-5e6b8c7a67a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:00.919709 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b9bae-e436-4cae-ac7d-5e6b8c7a67a4 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:00.962666 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 848/1705] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:00 2022] POST /identity/v3/users => generated 284 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:00.968167 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b9bae-e436-4cae-ac7d-5e6b8c7a67a4 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:00.982287 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b9bae-e436-4cae-ac7d-5e6b8c7a67a4 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:00.995310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef07fd-edea-4a1f-b7d3-41baaff1160d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:00.996418 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:00.999850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef07fd-edea-4a1f-b7d3-41baaff1160d admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.000148 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef07fd-edea-4a1f-b7d3-41baaff1160d admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.000518 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef07fd-edea-4a1f-b7d3-41baaff1160d admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.001288 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef07fd-edea-4a1f-b7d3-41baaff1160d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:01.009807 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef07fd-edea-4a1f-b7d3-41baaff1160d admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:01.031247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 849/1706] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:01.035444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef07fd-edea-4a1f-b7d3-41baaff1160d admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:01.040609 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e786a61-4a24-4f7e-84ff-bd097ee2027f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:01.042409 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e786a61-4a24-4f7e-84ff-bd097ee2027f service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.043476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e786a61-4a24-4f7e-84ff-bd097ee2027f service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.045421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e786a61-4a24-4f7e-84ff-bd097ee2027f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.058252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef07fd-edea-4a1f-b7d3-41baaff1160d admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:01.066610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa748d95-769d-45f2-9113-4397ce44f97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:01.067557 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa748d95-769d-45f2-9113-4397ce44f97c admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.067987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa748d95-769d-45f2-9113-4397ce44f97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.068484 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa748d95-769d-45f2-9113-4397ce44f97c admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.069382 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa748d95-769d-45f2-9113-4397ce44f97c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:01.071205 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa748d95-769d-45f2-9113-4397ce44f97c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:01.112325 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 850/1707] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:01.114588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59d055-64f9-42e3-93b9-c479850cbc76 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.115311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59d055-64f9-42e3-93b9-c479850cbc76 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.115768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59d055-64f9-42e3-93b9-c479850cbc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.189341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-8a59d055-64f9-42e3-93b9-c479850cbc76 None None] MFA Rules not processed for user `3b7b1014c466482b95820616b54dd13a`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:01.191116 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-8a59d055-64f9-42e3-93b9-c479850cbc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:01.222933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e786a61-4a24-4f7e-84ff-bd097ee2027f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:01.225064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e786a61-4a24-4f7e-84ff-bd097ee2027f service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:01.265385 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 858/1708] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:01.266415 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e786a61-4a24-4f7e-84ff-bd097ee2027f service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:01.291457 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e786a61-4a24-4f7e-84ff-bd097ee2027f service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:01.298745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8450ca-506b-4d07-9420-04791536c73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:01.300173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8450ca-506b-4d07-9420-04791536c73a admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.312183 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8450ca-506b-4d07-9420-04791536c73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.313178 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8450ca-506b-4d07-9420-04791536c73a admin admin] PATH_INFO: `/v3/projects/a72b7b54631842e4b2777eacf0a59fe8/users/6897188029014539a8c273a410c37af5/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.354325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8450ca-506b-4d07-9420-04791536c73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a72b7b54631842e4b2777eacf0a59fe8, user_id=6897188029014539a8c273a410c37af5, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:01.357497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8450ca-506b-4d07-9420-04791536c73a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:01.405056 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 859/1709] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:01 2022] PUT /identity/v3/projects/a72b7b54631842e4b2777eacf0a59fe8/users/6897188029014539a8c273a410c37af5/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:01.407602 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc3f91-8ece-407a-bc2c-c5ffc4e5d4f2 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.411542 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc3f91-8ece-407a-bc2c-c5ffc4e5d4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.412902 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc3f91-8ece-407a-bc2c-c5ffc4e5d4f2 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.419993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 860/1710] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jul 20 19:24:01 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:24:01.421401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc3f91-8ece-407a-bc2c-c5ffc4e5d4f2 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:01.429165 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-77fc3f91-8ece-407a-bc2c-c5ffc4e5d4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:01.435362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 851/1711] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:01.436788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd5007-7185-4985-90f4-79f858cbbefa None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.437226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd5007-7185-4985-90f4-79f858cbbefa None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.437504 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd5007-7185-4985-90f4-79f858cbbefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.486598 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-82fd5007-7185-4985-90f4-79f858cbbefa None None] MFA Rules not processed for user `6897188029014539a8c273a410c37af5`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:01.488542 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-82fd5007-7185-4985-90f4-79f858cbbefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:01.494381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:01.671433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 852/1712] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:01.685374 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59c6c9-7354-4055-944c-59282a7336f5 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.694311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59c6c9-7354-4055-944c-59282a7336f5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.695144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59c6c9-7354-4055-944c-59282a7336f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.713717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc3f91-8ece-407a-bc2c-c5ffc4e5d4f2 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:01.761789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f5091-7197-4163-871b-1208a94f3827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:01.763258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f5091-7197-4163-871b-1208a94f3827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.766374 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f5091-7197-4163-871b-1208a94f3827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.766516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f5091-7197-4163-871b-1208a94f3827 admin admin] PATH_INFO: `/v3/users/434a0e5365ad4af19201dd3f331e6a34` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.768066 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-5f59c6c9-7354-4055-944c-59282a7336f5 None None] MFA Rules not processed for user `3b7b1014c466482b95820616b54dd13a`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:01.769350 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5f59c6c9-7354-4055-944c-59282a7336f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:01.772170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f5091-7197-4163-871b-1208a94f3827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=434a0e5365ad4af19201dd3f331e6a34)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:01.774187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f5091-7197-4163-871b-1208a94f3827 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:01.946483 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0e4f5091-7197-4163-871b-1208a94f3827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '434a0e5365ad4af19201dd3f331e6a34'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:01.952812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 853/1713] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:01.960958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82917395-5c81-489a-83c3-942b1e2e19ae None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.961966 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0e4f5091-7197-4163-871b-1208a94f3827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '434a0e5365ad4af19201dd3f331e6a34'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:01.962248 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82917395-5c81-489a-83c3-942b1e2e19ae None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.962673 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82917395-5c81-489a-83c3-942b1e2e19ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.964487 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0e4f5091-7197-4163-871b-1208a94f3827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '434a0e5365ad4af19201dd3f331e6a34'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:01.969401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:01.985536 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0e4f5091-7197-4163-871b-1208a94f3827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '434a0e5365ad4af19201dd3f331e6a34'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:02.023496 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0e4f5091-7197-4163-871b-1208a94f3827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '434a0e5365ad4af19201dd3f331e6a34'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:02.028064 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-82917395-5c81-489a-83c3-942b1e2e19ae None None] MFA Rules not processed for user `cb06eeba738b452fab3986de7e1f7856`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:02.029596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-82917395-5c81-489a-83c3-942b1e2e19ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:02.040659 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 861/1714] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:01 2022] DELETE /identity/v3/users/434a0e5365ad4af19201dd3f331e6a34 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:02.045732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa249c6-3b3d-43f6-9dcc-abf689b8a2e6 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:02.046024 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa249c6-3b3d-43f6-9dcc-abf689b8a2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:02.046306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa249c6-3b3d-43f6-9dcc-abf689b8a2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:02.102507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-aaa249c6-3b3d-43f6-9dcc-abf689b8a2e6 None None] MFA Rules not processed for user `ae581d3dda354e07894cb5205d1b2c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:02.108280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-aaa249c6-3b3d-43f6-9dcc-abf689b8a2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:02.173556 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 854/1715] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:02.173556 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82917395-5c81-489a-83c3-942b1e2e19ae None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:02.173556 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-82917395-5c81-489a-83c3-942b1e2e19ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:02.248448 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82917395-5c81-489a-83c3-942b1e2e19ae None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:02.355061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 862/1716] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jul 20 19:24:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:02.355952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa249c6-3b3d-43f6-9dcc-abf689b8a2e6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:02.358747 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-aaa249c6-3b3d-43f6-9dcc-abf689b8a2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:02.461202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:02.483999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30626e57-6509-4926-b13d-73703bd42583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:02.485218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30626e57-6509-4926-b13d-73703bd42583 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:02.486158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30626e57-6509-4926-b13d-73703bd42583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:02.495927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30626e57-6509-4926-b13d-73703bd42583 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:02.498241 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30626e57-6509-4926-b13d-73703bd42583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:02.503038 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30626e57-6509-4926-b13d-73703bd42583 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:02.580356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 855/1717] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:02 2022] POST /identity/v3/projects => generated 365 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:02.584805 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30626e57-6509-4926-b13d-73703bd42583 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:02.643745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa249c6-3b3d-43f6-9dcc-abf689b8a2e6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:02.770286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30626e57-6509-4926-b13d-73703bd42583 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:02.781049 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ffce0-2b84-4c81-ad5b-65623cd329de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:02.785083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ffce0-2b84-4c81-ad5b-65623cd329de admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:02.785515 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ffce0-2b84-4c81-ad5b-65623cd329de admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:02.785903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ffce0-2b84-4c81-ad5b-65623cd329de admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:02.787034 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ffce0-2b84-4c81-ad5b-65623cd329de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:02.791612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ffce0-2b84-4c81-ad5b-65623cd329de admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:02.810015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb982925-9510-43d8-a13b-b4f8a67bbcda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:02.810015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb982925-9510-43d8-a13b-b4f8a67bbcda service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:02.810321 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb982925-9510-43d8-a13b-b4f8a67bbcda service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:02.810572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb982925-9510-43d8-a13b-b4f8a67bbcda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:02.813073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fb982925-9510-43d8-a13b-b4f8a67bbcda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:02.819788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:02.843456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 856/1718] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:02 2022] POST /identity/v3/users => generated 289 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:02.843456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ffce0-2b84-4c81-ad5b-65623cd329de admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:02.849674 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-122ffce0-2b84-4c81-ad5b-65623cd329de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:02.947263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb982925-9510-43d8-a13b-b4f8a67bbcda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:02.949049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb982925-9510-43d8-a13b-b4f8a67bbcda service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:02.974448 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 863/1719] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:02.975935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb982925-9510-43d8-a13b-b4f8a67bbcda service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:03.001677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb982925-9510-43d8-a13b-b4f8a67bbcda service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:03.020159 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ead822-3eac-41d7-8121-bd0b20939a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:03.022115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ead822-3eac-41d7-8121-bd0b20939a95 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:03.022411 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ead822-3eac-41d7-8121-bd0b20939a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:03.025017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ead822-3eac-41d7-8121-bd0b20939a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:03.028597 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ead822-3eac-41d7-8121-bd0b20939a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:03.030800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ead822-3eac-41d7-8121-bd0b20939a95 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:03.068059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 864/1720] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:03.072249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ead822-3eac-41d7-8121-bd0b20939a95 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:03.091856 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ffce0-2b84-4c81-ad5b-65623cd329de admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:03.094963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ead822-3eac-41d7-8121-bd0b20939a95 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:03.105625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99b30d-6559-4670-ae7a-0427ab313b6c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae581d3dda354e07894cb5205d1b2c5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:03.107680 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f7f28-aa30-422e-bfb7-e97dfccb7b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:03.108779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99b30d-6559-4670-ae7a-0427ab313b6c service placement] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:03.109073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99b30d-6559-4670-ae7a-0427ab313b6c service placement] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:03.109367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99b30d-6559-4670-ae7a-0427ab313b6c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:03.109833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f7f28-aa30-422e-bfb7-e97dfccb7b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:03.110119 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f7f28-aa30-422e-bfb7-e97dfccb7b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:03.110924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f7f28-aa30-422e-bfb7-e97dfccb7b8c admin admin] PATH_INFO: `/v3/projects/09ceaa7b34b149b3a62be8401e194981/users/908b5c7a110e43e89156ebca1ed54ed8/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:03.136748 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99b30d-6559-4670-ae7a-0427ab313b6c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:03.143154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99b30d-6559-4670-ae7a-0427ab313b6c service placement] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:03.154168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f7f28-aa30-422e-bfb7-e97dfccb7b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=09ceaa7b34b149b3a62be8401e194981, user_id=908b5c7a110e43e89156ebca1ed54ed8, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:03.156893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f7f28-aa30-422e-bfb7-e97dfccb7b8c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:03.169229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 857/1721] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:03.170113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99b30d-6559-4670-ae7a-0427ab313b6c service placement] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:03.175054 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4a99b30d-6559-4670-ae7a-0427ab313b6c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:03.199196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 865/1722] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:03 2022] PUT /identity/v3/projects/09ceaa7b34b149b3a62be8401e194981/users/908b5c7a110e43e89156ebca1ed54ed8/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:03.200365 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f7f28-aa30-422e-bfb7-e97dfccb7b8c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:03.213182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99b30d-6559-4670-ae7a-0427ab313b6c service placement] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:03.375277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f7f28-aa30-422e-bfb7-e97dfccb7b8c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:03.381627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6306cc-32ba-4bf4-9160-7923a016d7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:03.382162 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6306cc-32ba-4bf4-9160-7923a016d7cc admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:03.382604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6306cc-32ba-4bf4-9160-7923a016d7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:03.382762 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6306cc-32ba-4bf4-9160-7923a016d7cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:03.383571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6306cc-32ba-4bf4-9160-7923a016d7cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:03.385665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6306cc-32ba-4bf4-9160-7923a016d7cc admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:03.407270 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 866/1723] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:03.408607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24e38d-90cb-4742-8999-76034280479b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:03.408903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24e38d-90cb-4742-8999-76034280479b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:03.409189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24e38d-90cb-4742-8999-76034280479b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:03.413844 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33532d-edf9-4287-a0bd-26b6ec48c742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:03.415238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33532d-edf9-4287-a0bd-26b6ec48c742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:03.415528 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33532d-edf9-4287-a0bd-26b6ec48c742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:03.415810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33532d-edf9-4287-a0bd-26b6ec48c742 admin admin] PATH_INFO: `/v3/projects/ad44b110b66740669f1360228ac3e901` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:03.418878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33532d-edf9-4287-a0bd-26b6ec48c742 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad44b110b66740669f1360228ac3e901)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:03.423413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33532d-edf9-4287-a0bd-26b6ec48c742 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:03.428080 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:03.437046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7b24e38d-90cb-4742-8999-76034280479b None None] MFA Rules not processed for user `74327a10eaab43e0af02050da1a24f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:03.440809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7b24e38d-90cb-4742-8999-76034280479b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:03.544230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 867/1724] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jul 20 19:24:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:03.545145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24e38d-90cb-4742-8999-76034280479b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:03.546709 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ab33532d-edf9-4287-a0bd-26b6ec48c742 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad44b110b66740669f1360228ac3e901', 'request_id': 'req-ab33532d-edf9-4287-a0bd-26b6ec48c742'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:03.561139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24e38d-90cb-4742-8999-76034280479b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:03.571517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ab33532d-edf9-4287-a0bd-26b6ec48c742 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad44b110b66740669f1360228ac3e901', 'request_id': 'req-ab33532d-edf9-4287-a0bd-26b6ec48c742'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:03.580420 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:03.582396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ab33532d-edf9-4287-a0bd-26b6ec48c742 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad44b110b66740669f1360228ac3e901', 'request_id': 'req-ab33532d-edf9-4287-a0bd-26b6ec48c742'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:03.619305 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ab33532d-edf9-4287-a0bd-26b6ec48c742 admin admin] The token cache is being invalidate because project ad44b110b66740669f1360228ac3e901 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:03.619645 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ab33532d-edf9-4287-a0bd-26b6ec48c742 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:03.623650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ab33532d-edf9-4287-a0bd-26b6ec48c742 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:03.628879 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 858/1725] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:03 2022] DELETE /identity/v3/projects/ad44b110b66740669f1360228ac3e901 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:03.628879 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33532d-edf9-4287-a0bd-26b6ec48c742 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:03.632379 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ab33532d-edf9-4287-a0bd-26b6ec48c742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:03.659326 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b6624-7827-4e21-89bf-d6330947330a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:03.661528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b6624-7827-4e21-89bf-d6330947330a admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:03.661818 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b6624-7827-4e21-89bf-d6330947330a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:03.662125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b6624-7827-4e21-89bf-d6330947330a admin admin] PATH_INFO: `/v3/projects/09ceaa7b34b149b3a62be8401e194981/users/908b5c7a110e43e89156ebca1ed54ed8/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:03.665089 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b6624-7827-4e21-89bf-d6330947330a admin admin] RBAC: Authorizing `identity:create_grant(project_id=09ceaa7b34b149b3a62be8401e194981, user_id=908b5c7a110e43e89156ebca1ed54ed8, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:03.674489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b6624-7827-4e21-89bf-d6330947330a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:03.696183 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 868/1726] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:03 2022] PUT /identity/v3/projects/09ceaa7b34b149b3a62be8401e194981/users/908b5c7a110e43e89156ebca1ed54ed8/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:03.696183 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b6624-7827-4e21-89bf-d6330947330a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:03.697641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ec1b6624-7827-4e21-89bf-d6330947330a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:03.836170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b6624-7827-4e21-89bf-d6330947330a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:03.844835 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307e401-b831-4889-8a78-546502989aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:03.845804 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307e401-b831-4889-8a78-546502989aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:03.846094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307e401-b831-4889-8a78-546502989aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:03.846376 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307e401-b831-4889-8a78-546502989aa0 admin admin] PATH_INFO: `/v3/users/831a941b614f4ed7b3c9af6818b4c4cd` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:03.849357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307e401-b831-4889-8a78-546502989aa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=831a941b614f4ed7b3c9af6818b4c4cd)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:03.850759 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307e401-b831-4889-8a78-546502989aa0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:03.859873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33532d-edf9-4287-a0bd-26b6ec48c742 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:03.923955 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4307e401-b831-4889-8a78-546502989aa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '831a941b614f4ed7b3c9af6818b4c4cd'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:03.938479 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4307e401-b831-4889-8a78-546502989aa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '831a941b614f4ed7b3c9af6818b4c4cd'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:03.940197 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4307e401-b831-4889-8a78-546502989aa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '831a941b614f4ed7b3c9af6818b4c4cd'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:03.954608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4307e401-b831-4889-8a78-546502989aa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '831a941b614f4ed7b3c9af6818b4c4cd'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:03.971026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4307e401-b831-4889-8a78-546502989aa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '831a941b614f4ed7b3c9af6818b4c4cd'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:03.981348 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 869/1727] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:03 2022] DELETE /identity/v3/users/831a941b614f4ed7b3c9af6818b4c4cd => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:03.981862 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b7a3a-c516-4917-a8e9-f65eece3efb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:03.982536 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b7a3a-c516-4917-a8e9-f65eece3efb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:03.982819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b7a3a-c516-4917-a8e9-f65eece3efb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:03.983100 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b7a3a-c516-4917-a8e9-f65eece3efb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:03.983896 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e2815-8cb3-4f80-86c7-c463d040647e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:03.984412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e2815-8cb3-4f80-86c7-c463d040647e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:03.984881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e2815-8cb3-4f80-86c7-c463d040647e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:03.986296 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-151b7a3a-c516-4917-a8e9-f65eece3efb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:04.020982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b7a3a-c516-4917-a8e9-f65eece3efb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:04.021864 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-df8e2815-8cb3-4f80-86c7-c463d040647e None None] MFA Rules not processed for user `908b5c7a110e43e89156ebca1ed54ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:04.022423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b7a3a-c516-4917-a8e9-f65eece3efb0 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:04.023500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-df8e2815-8cb3-4f80-86c7-c463d040647e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:04.041963 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 859/1728] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:04.052735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:04.129868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 870/1729] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:04.141463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b7a3a-c516-4917-a8e9-f65eece3efb0 service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:04.143520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-151b7a3a-c516-4917-a8e9-f65eece3efb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:04.239977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:04.244641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9f00d-d22f-4d6c-99d3-d7feb505b02c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:04.245499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9f00d-d22f-4d6c-99d3-d7feb505b02c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:04.245499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9f00d-d22f-4d6c-99d3-d7feb505b02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:04.247546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b7a3a-c516-4917-a8e9-f65eece3efb0 service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:04.310647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-bdc9f00d-d22f-4d6c-99d3-d7feb505b02c None None] MFA Rules not processed for user `9ac790383bdf4eecaef96cc25fe38553`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:04.312673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bdc9f00d-d22f-4d6c-99d3-d7feb505b02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:04.348247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eeecea-3feb-44d6-9cab-6115f761bebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:04.351808 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eeecea-3feb-44d6-9cab-6115f761bebd service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:04.352254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eeecea-3feb-44d6-9cab-6115f761bebd service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:04.352915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eeecea-3feb-44d6-9cab-6115f761bebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:04.395119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eeecea-3feb-44d6-9cab-6115f761bebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:04.401753 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eeecea-3feb-44d6-9cab-6115f761bebd service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:04.442904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 860/1730] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:04.492376 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eeecea-3feb-44d6-9cab-6115f761bebd service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:04.494313 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c5eeecea-3feb-44d6-9cab-6115f761bebd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:04.549200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 871/1731] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 20 19:24:04.551086 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9f00d-d22f-4d6c-99d3-d7feb505b02c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:04.575260 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bdc9f00d-d22f-4d6c-99d3-d7feb505b02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:04.623223 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9f00d-d22f-4d6c-99d3-d7feb505b02c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:04.627774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08303301-ad46-4191-99dc-6c9799736e06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:04.640355 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08303301-ad46-4191-99dc-6c9799736e06 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:04.640914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08303301-ad46-4191-99dc-6c9799736e06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:04.641336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08303301-ad46-4191-99dc-6c9799736e06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:04.652014 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-08303301-ad46-4191-99dc-6c9799736e06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:04.762399 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eeecea-3feb-44d6-9cab-6115f761bebd service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:04.780559 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd6cfe-389b-43d0-aeae-0aba6ea1aa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:04.781759 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd6cfe-389b-43d0-aeae-0aba6ea1aa6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:04.782290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd6cfe-389b-43d0-aeae-0aba6ea1aa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:04.782764 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd6cfe-389b-43d0-aeae-0aba6ea1aa6f admin admin] PATH_INFO: `/v3/projects/b67255a6107044a18b4f70f4bd95e0e2` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:04.784212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd6cfe-389b-43d0-aeae-0aba6ea1aa6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b67255a6107044a18b4f70f4bd95e0e2)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:04.786495 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd6cfe-389b-43d0-aeae-0aba6ea1aa6f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:04.885332 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08303301-ad46-4191-99dc-6c9799736e06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:04.886803 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08303301-ad46-4191-99dc-6c9799736e06 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:04.902947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 872/1732] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:04 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:04.903829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08303301-ad46-4191-99dc-6c9799736e06 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:04.905507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-08303301-ad46-4191-99dc-6c9799736e06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:04.910788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-62fd6cfe-389b-43d0-aeae-0aba6ea1aa6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b67255a6107044a18b4f70f4bd95e0e2', 'request_id': 'req-62fd6cfe-389b-43d0-aeae-0aba6ea1aa6f'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:04.936891 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-62fd6cfe-389b-43d0-aeae-0aba6ea1aa6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b67255a6107044a18b4f70f4bd95e0e2', 'request_id': 'req-62fd6cfe-389b-43d0-aeae-0aba6ea1aa6f'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:04.952067 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-62fd6cfe-389b-43d0-aeae-0aba6ea1aa6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b67255a6107044a18b4f70f4bd95e0e2', 'request_id': 'req-62fd6cfe-389b-43d0-aeae-0aba6ea1aa6f'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:04.989137 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-62fd6cfe-389b-43d0-aeae-0aba6ea1aa6f admin admin] The token cache is being invalidate because project b67255a6107044a18b4f70f4bd95e0e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:04.989851 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-62fd6cfe-389b-43d0-aeae-0aba6ea1aa6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:04.993508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-62fd6cfe-389b-43d0-aeae-0aba6ea1aa6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:05.009394 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 861/1733] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:04 2022] DELETE /identity/v3/projects/b67255a6107044a18b4f70f4bd95e0e2 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:05.014971 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac1e3a-431b-4f39-ace5-55fdf9e1a896 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:05.015288 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac1e3a-431b-4f39-ace5-55fdf9e1a896 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:05.015547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac1e3a-431b-4f39-ace5-55fdf9e1a896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:05.046545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:05.063895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-82ac1e3a-431b-4f39-ace5-55fdf9e1a896 None None] MFA Rules not processed for user `b15f8d66d5104aaeae51e6c1fa3b3596`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:05.065495 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-82ac1e3a-431b-4f39-ace5-55fdf9e1a896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:05.138706 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:05.189141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08303301-ad46-4191-99dc-6c9799736e06 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:05.269549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 862/1734] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:05.269549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f6076-92ba-44a2-8c3b-f248230da4b1 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:05.269549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f6076-92ba-44a2-8c3b-f248230da4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:05.269549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f6076-92ba-44a2-8c3b-f248230da4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:05.339725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-201f6076-92ba-44a2-8c3b-f248230da4b1 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:05.346642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-201f6076-92ba-44a2-8c3b-f248230da4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:05.375455 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2aa86-bd1f-4bec-9afd-e3922c67d1ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:05.379333 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2aa86-bd1f-4bec-9afd-e3922c67d1ab service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:05.381842 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2aa86-bd1f-4bec-9afd-e3922c67d1ab service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:05.383387 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2aa86-bd1f-4bec-9afd-e3922c67d1ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:05.389379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-04a2aa86-bd1f-4bec-9afd-e3922c67d1ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:05.573607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 863/1735] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:05.577854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f6076-92ba-44a2-8c3b-f248230da4b1 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:05.579326 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-201f6076-92ba-44a2-8c3b-f248230da4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:05.609610 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2aa86-bd1f-4bec-9afd-e3922c67d1ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:05.611332 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2aa86-bd1f-4bec-9afd-e3922c67d1ab service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:05.659332 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f6076-92ba-44a2-8c3b-f248230da4b1 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:05.680510 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8a1f7-3846-4dce-978c-83691cb9ecc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:05.682916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8a1f7-3846-4dce-978c-83691cb9ecc2 service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:05.683831 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8a1f7-3846-4dce-978c-83691cb9ecc2 service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:05.684568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8a1f7-3846-4dce-978c-83691cb9ecc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:05.704677 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8a1f7-3846-4dce-978c-83691cb9ecc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:05.704868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 873/1736] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 801 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:05.707000 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8a1f7-3846-4dce-978c-83691cb9ecc2 service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:05.718022 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da102480-518c-4eb3-85b2-df2572f5261c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:05.718329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da102480-518c-4eb3-85b2-df2572f5261c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:05.722477 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da102480-518c-4eb3-85b2-df2572f5261c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:05.750418 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 864/1737] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:05.832354 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-da102480-518c-4eb3-85b2-df2572f5261c None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:05.836018 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-da102480-518c-4eb3-85b2-df2572f5261c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:05.858534 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 874/1738] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:05.862279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8a1f7-3846-4dce-978c-83691cb9ecc2 service nova] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:05.878671 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8a1f7-3846-4dce-978c-83691cb9ecc2 service nova] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:05.911126 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c1932-7df8-4563-8ce9-49b031b239e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:05.911934 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c1932-7df8-4563-8ce9-49b031b239e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:05.912401 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c1932-7df8-4563-8ce9-49b031b239e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:05.912807 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c1932-7df8-4563-8ce9-49b031b239e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:05.913892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c1932-7df8-4563-8ce9-49b031b239e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:05.913892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c1932-7df8-4563-8ce9-49b031b239e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:05.916437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c1932-7df8-4563-8ce9-49b031b239e2 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:05.964091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da102480-518c-4eb3-85b2-df2572f5261c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:05.965599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-da102480-518c-4eb3-85b2-df2572f5261c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:05.975726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 865/1739] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:05.986585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c1932-7df8-4563-8ce9-49b031b239e2 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:06.025328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c1932-7df8-4563-8ce9-49b031b239e2 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:06.051074 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a16390-3f33-4a89-872c-99f1f1a393b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:06.055088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a16390-3f33-4a89-872c-99f1f1a393b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:06.055256 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a16390-3f33-4a89-872c-99f1f1a393b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:06.055441 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a16390-3f33-4a89-872c-99f1f1a393b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:06.059278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a16390-3f33-4a89-872c-99f1f1a393b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:06.065968 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a16390-3f33-4a89-872c-99f1f1a393b4 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:06.148857 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 866/1740] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:05 2022] POST /identity/v3/projects => generated 401 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:06.153236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a16390-3f33-4a89-872c-99f1f1a393b4 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:06.170321 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da102480-518c-4eb3-85b2-df2572f5261c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:06.235557 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:06.324978 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fd9f1-257d-4811-afcc-1ba0c1a61a36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:06.325645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fd9f1-257d-4811-afcc-1ba0c1a61a36 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:06.340897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fd9f1-257d-4811-afcc-1ba0c1a61a36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:06.341278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fd9f1-257d-4811-afcc-1ba0c1a61a36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:06.353758 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a16390-3f33-4a89-872c-99f1f1a393b4 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:06.357349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f10fd9f1-257d-4811-afcc-1ba0c1a61a36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:06.375468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6df91a-cdb1-4e1c-8534-e56d96b45f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:06.383163 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6df91a-cdb1-4e1c-8534-e56d96b45f1c admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:06.386371 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6df91a-cdb1-4e1c-8534-e56d96b45f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:06.386615 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6df91a-cdb1-4e1c-8534-e56d96b45f1c admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:06.387368 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6df91a-cdb1-4e1c-8534-e56d96b45f1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:06.403024 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6df91a-cdb1-4e1c-8534-e56d96b45f1c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:06.478697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 867/1741] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:06 2022] POST /identity/v3/users => generated 301 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:06.482753 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6df91a-cdb1-4e1c-8534-e56d96b45f1c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:06.519154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6df91a-cdb1-4e1c-8534-e56d96b45f1c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:06.548029 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73c699-fea7-4abe-8b79-4c732c420c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:06.555588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73c699-fea7-4abe-8b79-4c732c420c83 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:06.555904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73c699-fea7-4abe-8b79-4c732c420c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:06.556187 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73c699-fea7-4abe-8b79-4c732c420c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:06.565515 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73c699-fea7-4abe-8b79-4c732c420c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:06.576428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73c699-fea7-4abe-8b79-4c732c420c83 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:06.590278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:06.636055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 868/1742] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:06.643614 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73c699-fea7-4abe-8b79-4c732c420c83 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:06.681750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73c699-fea7-4abe-8b79-4c732c420c83 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:06.691154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10fd9f1-257d-4811-afcc-1ba0c1a61a36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:06.694468 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10fd9f1-257d-4811-afcc-1ba0c1a61a36 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:06.697835 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45404f6f-44ed-480d-8e0a-614f69ae4db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:06.719309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45404f6f-44ed-480d-8e0a-614f69ae4db4 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:06.721359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45404f6f-44ed-480d-8e0a-614f69ae4db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:06.722349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45404f6f-44ed-480d-8e0a-614f69ae4db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:06.728539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45404f6f-44ed-480d-8e0a-614f69ae4db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:06.738458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 875/1743] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:05 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 775 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:06.744544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f3c0a-918d-4866-9c81-182218bebb55 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:06.744915 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f3c0a-918d-4866-9c81-182218bebb55 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:06.747390 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f3c0a-918d-4866-9c81-182218bebb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:06.754506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45404f6f-44ed-480d-8e0a-614f69ae4db4 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:06.831527 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 869/1744] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:06.836731 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45404f6f-44ed-480d-8e0a-614f69ae4db4 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:06.880476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45404f6f-44ed-480d-8e0a-614f69ae4db4 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:06.899939 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02caa8-ef48-4a23-99f3-407cc5294f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:06.905078 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02caa8-ef48-4a23-99f3-407cc5294f75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:06.905371 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02caa8-ef48-4a23-99f3-407cc5294f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:06.905660 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02caa8-ef48-4a23-99f3-407cc5294f75 admin admin] PATH_INFO: `/v3/projects/fb22045070644bfc9a3574ad6b004a81/users/0be76125f91745bb8d6d723ac8adfae2/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:06.942920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ce1f3c0a-918d-4866-9c81-182218bebb55 None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:06.945859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ce1f3c0a-918d-4866-9c81-182218bebb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:06.975061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02caa8-ef48-4a23-99f3-407cc5294f75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb22045070644bfc9a3574ad6b004a81, user_id=0be76125f91745bb8d6d723ac8adfae2, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:06.980056 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02caa8-ef48-4a23-99f3-407cc5294f75 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:07.043438 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 870/1745] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:06 2022] PUT /identity/v3/projects/fb22045070644bfc9a3574ad6b004a81/users/0be76125f91745bb8d6d723ac8adfae2/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:07.060384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4cc8a-306d-4562-8827-aeb244544b8a None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:07.064164 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4cc8a-306d-4562-8827-aeb244544b8a None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:07.064528 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4cc8a-306d-4562-8827-aeb244544b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:07.246780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-aaa4cc8a-306d-4562-8827-aeb244544b8a None None] MFA Rules not processed for user `0be76125f91745bb8d6d723ac8adfae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:07.267810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-aaa4cc8a-306d-4562-8827-aeb244544b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:07.313405 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 876/1746] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jul 20 19:24:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 574 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:07.459722 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 871/1747] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:07.461266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f3c0a-918d-4866-9c81-182218bebb55 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:07.479315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f3c0a-918d-4866-9c81-182218bebb55 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:07.673565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4cc8a-306d-4562-8827-aeb244544b8a None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:07.674862 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-aaa4cc8a-306d-4562-8827-aeb244544b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:07.704998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:07.766104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410dfbe6-f1c9-433c-a8e3-8be5d9b6e9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:07.766801 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410dfbe6-f1c9-433c-a8e3-8be5d9b6e9ca admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:07.767087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410dfbe6-f1c9-433c-a8e3-8be5d9b6e9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:07.767371 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410dfbe6-f1c9-433c-a8e3-8be5d9b6e9ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:07.768116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410dfbe6-f1c9-433c-a8e3-8be5d9b6e9ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:07.779560 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410dfbe6-f1c9-433c-a8e3-8be5d9b6e9ca admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:07.898061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 877/1748] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:07 2022] POST /identity/v3/projects => generated 401 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:07.900949 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410dfbe6-f1c9-433c-a8e3-8be5d9b6e9ca admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:08.190356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4cc8a-306d-4562-8827-aeb244544b8a None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:08.236716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410dfbe6-f1c9-433c-a8e3-8be5d9b6e9ca admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:08.266447 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a08545-9c20-4eac-a902-8b86dca06c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:08.271815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a08545-9c20-4eac-a902-8b86dca06c86 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:08.272617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a08545-9c20-4eac-a902-8b86dca06c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:08.275323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a08545-9c20-4eac-a902-8b86dca06c86 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:08.279976 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a08545-9c20-4eac-a902-8b86dca06c86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:08.285603 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a08545-9c20-4eac-a902-8b86dca06c86 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:08.345651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 878/1749] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:07 2022] POST /identity/v3/users => generated 307 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:08.350323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a08545-9c20-4eac-a902-8b86dca06c86 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:08.406765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:08.411539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a08545-9c20-4eac-a902-8b86dca06c86 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:08.436721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ebed6-6c7a-460c-8ff0-5b6d391c77a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:08.438748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ebed6-6c7a-460c-8ff0-5b6d391c77a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:08.443566 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ebed6-6c7a-460c-8ff0-5b6d391c77a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:08.443912 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ebed6-6c7a-460c-8ff0-5b6d391c77a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:08.447897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ebed6-6c7a-460c-8ff0-5b6d391c77a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:08.458502 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ebed6-6c7a-460c-8ff0-5b6d391c77a2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:08.507354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616ffe6-6898-443e-a2e2-9d57de0afe07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:08.508908 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616ffe6-6898-443e-a2e2-9d57de0afe07 service swift] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:08.509694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616ffe6-6898-443e-a2e2-9d57de0afe07 service swift] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:08.509694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616ffe6-6898-443e-a2e2-9d57de0afe07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:08.532761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 879/1750] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:08.533735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ebed6-6c7a-460c-8ff0-5b6d391c77a2 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:08.590337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ebed6-6c7a-460c-8ff0-5b6d391c77a2 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:08.634627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1915e2-6e13-492b-ae30-69e6cdff508a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:08.639028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1915e2-6e13-492b-ae30-69e6cdff508a admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:08.640423 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1915e2-6e13-492b-ae30-69e6cdff508a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:08.641650 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1915e2-6e13-492b-ae30-69e6cdff508a admin admin] PATH_INFO: `/v3/projects/fcc0de8adfe547d999d5d45c83b682dc/users/596600e15d2649d9b47ed82ce7633d4d/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:08.647364 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616ffe6-6898-443e-a2e2-9d57de0afe07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:08.649283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616ffe6-6898-443e-a2e2-9d57de0afe07 service swift] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:08.675680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 872/1751] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:24:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1003 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:24:08.694390 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1915e2-6e13-492b-ae30-69e6cdff508a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcc0de8adfe547d999d5d45c83b682dc, user_id=596600e15d2649d9b47ed82ce7633d4d, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:08.706873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1915e2-6e13-492b-ae30-69e6cdff508a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:08.767348 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 880/1752] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:08 2022] PUT /identity/v3/projects/fcc0de8adfe547d999d5d45c83b682dc/users/596600e15d2649d9b47ed82ce7633d4d/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:08.771464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616ffe6-6898-443e-a2e2-9d57de0afe07 service swift] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:08.796808 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616ffe6-6898-443e-a2e2-9d57de0afe07 service swift] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:09.078838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cc51b-5d98-4cc5-a471-ccab336b112a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:09.079651 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cc51b-5d98-4cc5-a471-ccab336b112a admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:09.080138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cc51b-5d98-4cc5-a471-ccab336b112a admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:09.080521 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cc51b-5d98-4cc5-a471-ccab336b112a admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:09.081436 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59cc51b-5d98-4cc5-a471-ccab336b112a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:09.083548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59cc51b-5d98-4cc5-a471-ccab336b112a admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:09.127768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 873/1753] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:09.131521 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1915e2-6e13-492b-ae30-69e6cdff508a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:09.185389 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1915e2-6e13-492b-ae30-69e6cdff508a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:09.200188 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910ac55-70b5-4753-9880-c66a1e5c87ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:09.201940 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910ac55-70b5-4753-9880-c66a1e5c87ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:09.204096 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910ac55-70b5-4753-9880-c66a1e5c87ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:09.207315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910ac55-70b5-4753-9880-c66a1e5c87ee admin admin] PATH_INFO: `/v3/projects/fcc0de8adfe547d999d5d45c83b682dc/users/596600e15d2649d9b47ed82ce7633d4d/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:09.221037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910ac55-70b5-4753-9880-c66a1e5c87ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcc0de8adfe547d999d5d45c83b682dc, user_id=596600e15d2649d9b47ed82ce7633d4d, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:09.231154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910ac55-70b5-4753-9880-c66a1e5c87ee admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:09.307498 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 881/1754] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:09 2022] PUT /identity/v3/projects/fcc0de8adfe547d999d5d45c83b682dc/users/596600e15d2649d9b47ed82ce7633d4d/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:09.308793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10884b8a-4653-4b68-b285-1e7898e27b16 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:09.309655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10884b8a-4653-4b68-b285-1e7898e27b16 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:09.310093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10884b8a-4653-4b68-b285-1e7898e27b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:09.371014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-10884b8a-4653-4b68-b285-1e7898e27b16 None None] MFA Rules not processed for user `596600e15d2649d9b47ed82ce7633d4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:09.376565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-10884b8a-4653-4b68-b285-1e7898e27b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:09.559785 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:09.588524 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 874/1755] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:09 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:09.596275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910ac55-70b5-4753-9880-c66a1e5c87ee admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:09.599443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a910ac55-70b5-4753-9880-c66a1e5c87ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:09.744628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910ac55-70b5-4753-9880-c66a1e5c87ee admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:09.750789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80924977-d2c4-45c3-b52c-4d313c235104 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:09.751465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80924977-d2c4-45c3-b52c-4d313c235104 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:09.751749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80924977-d2c4-45c3-b52c-4d313c235104 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:09.752036 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80924977-d2c4-45c3-b52c-4d313c235104 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:09.778575 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80924977-d2c4-45c3-b52c-4d313c235104 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:09.780043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80924977-d2c4-45c3-b52c-4d313c235104 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:09.812539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 882/1756] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:09 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:10.116954 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10884b8a-4653-4b68-b285-1e7898e27b16 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:10.136953 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10884b8a-4653-4b68-b285-1e7898e27b16 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:10.164899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9576d-6bea-453f-8840-3e957d1b1ff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:10.167104 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9576d-6bea-453f-8840-3e957d1b1ff6 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:10.167691 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9576d-6bea-453f-8840-3e957d1b1ff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:10.167872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9576d-6bea-453f-8840-3e957d1b1ff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:10.564550 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9576d-6bea-453f-8840-3e957d1b1ff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:10.564550 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9576d-6bea-453f-8840-3e957d1b1ff6 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:10.608343 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 875/1757] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:11.892627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db7fb6-a828-40cc-8e85-c4b4d0700fdc None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:11.893491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db7fb6-a828-40cc-8e85-c4b4d0700fdc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:11.893784 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db7fb6-a828-40cc-8e85-c4b4d0700fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:11.971916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-63db7fb6-a828-40cc-8e85-c4b4d0700fdc None None] MFA Rules not processed for user `74327a10eaab43e0af02050da1a24f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:11.979051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-63db7fb6-a828-40cc-8e85-c4b4d0700fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:12.058403 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:12.180900 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9576d-6bea-453f-8840-3e957d1b1ff6 service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:12.182980 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-82a9576d-6bea-453f-8840-3e957d1b1ff6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:12.277138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 883/1758] 10.222.0.89 () {58 vars in 1015 bytes} [Wed Jul 20 19:24:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:12.315478 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db7fb6-a828-40cc-8e85-c4b4d0700fdc None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:12.317798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-63db7fb6-a828-40cc-8e85-c4b4d0700fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:12.350859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db7fb6-a828-40cc-8e85-c4b4d0700fdc None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:12.353771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44c055-40bc-4428-955d-157fe8a7f205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:12.356639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44c055-40bc-4428-955d-157fe8a7f205 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:12.356874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44c055-40bc-4428-955d-157fe8a7f205 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:12.357501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44c055-40bc-4428-955d-157fe8a7f205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:12.367984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44c055-40bc-4428-955d-157fe8a7f205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:12.371513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44c055-40bc-4428-955d-157fe8a7f205 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:12.393866 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 884/1759] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:12.429576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9576d-6bea-453f-8840-3e957d1b1ff6 service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:12.436897 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208165a-5284-4b2d-8097-c527db2d2f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:12.438776 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208165a-5284-4b2d-8097-c527db2d2f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:12.439133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208165a-5284-4b2d-8097-c527db2d2f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:12.439599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208165a-5284-4b2d-8097-c527db2d2f52 admin admin] PATH_INFO: `/v3/users/6897188029014539a8c273a410c37af5` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:12.441214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208165a-5284-4b2d-8097-c527db2d2f52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6897188029014539a8c273a410c37af5)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:12.445686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208165a-5284-4b2d-8097-c527db2d2f52 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:12.576236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-0208165a-5284-4b2d-8097-c527db2d2f52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6897188029014539a8c273a410c37af5'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:12.594823 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-0208165a-5284-4b2d-8097-c527db2d2f52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6897188029014539a8c273a410c37af5'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:12.638348 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-0208165a-5284-4b2d-8097-c527db2d2f52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6897188029014539a8c273a410c37af5'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:12.670524 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-0208165a-5284-4b2d-8097-c527db2d2f52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6897188029014539a8c273a410c37af5'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:12.672982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-0208165a-5284-4b2d-8097-c527db2d2f52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6897188029014539a8c273a410c37af5'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:12.698168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 876/1760] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:12 2022] DELETE /identity/v3/users/6897188029014539a8c273a410c37af5 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:12.705078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab017268-421e-4720-8cfb-c8ba8fa6f5f4 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:12.705566 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab017268-421e-4720-8cfb-c8ba8fa6f5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:12.706033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab017268-421e-4720-8cfb-c8ba8fa6f5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:12.753256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ab017268-421e-4720-8cfb-c8ba8fa6f5f4 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:12.754809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ab017268-421e-4720-8cfb-c8ba8fa6f5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:12.934474 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 885/1761] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:13.030108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208165a-5284-4b2d-8097-c527db2d2f52 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:13.032062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0208165a-5284-4b2d-8097-c527db2d2f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:13.131920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:13.281317 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208165a-5284-4b2d-8097-c527db2d2f52 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:13.302611 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed807a-7492-40d5-947f-2ddae0fcf327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:13.312420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed807a-7492-40d5-947f-2ddae0fcf327 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:13.313447 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed807a-7492-40d5-947f-2ddae0fcf327 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:13.313447 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed807a-7492-40d5-947f-2ddae0fcf327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:13.367316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed807a-7492-40d5-947f-2ddae0fcf327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:13.372255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed807a-7492-40d5-947f-2ddae0fcf327 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:13.413136 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab017268-421e-4720-8cfb-c8ba8fa6f5f4 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:13.414610 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ab017268-421e-4720-8cfb-c8ba8fa6f5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:13.466216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 877/1762] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:13.475024 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab017268-421e-4720-8cfb-c8ba8fa6f5f4 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:13.512763 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c217795-4110-40bc-a7ea-ce21df7f49cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:13.513713 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c217795-4110-40bc-a7ea-ce21df7f49cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:13.513996 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c217795-4110-40bc-a7ea-ce21df7f49cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:13.514271 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c217795-4110-40bc-a7ea-ce21df7f49cb admin admin] PATH_INFO: `/v3/users/5e1a1407ccb448b986dec140237f4b8d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:13.519211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c217795-4110-40bc-a7ea-ce21df7f49cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e1a1407ccb448b986dec140237f4b8d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:13.532235 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c217795-4110-40bc-a7ea-ce21df7f49cb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:13.715567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7c217795-4110-40bc-a7ea-ce21df7f49cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e1a1407ccb448b986dec140237f4b8d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:13.732969 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7c217795-4110-40bc-a7ea-ce21df7f49cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e1a1407ccb448b986dec140237f4b8d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:13.734000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7c217795-4110-40bc-a7ea-ce21df7f49cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e1a1407ccb448b986dec140237f4b8d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:13.758186 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7c217795-4110-40bc-a7ea-ce21df7f49cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e1a1407ccb448b986dec140237f4b8d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:13.802668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7c217795-4110-40bc-a7ea-ce21df7f49cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e1a1407ccb448b986dec140237f4b8d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:13.822145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 886/1763] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:13 2022] DELETE /identity/v3/users/5e1a1407ccb448b986dec140237f4b8d => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:13.827206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61b5ff-80a7-4420-8f84-baf2fc87821c None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:13.827570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61b5ff-80a7-4420-8f84-baf2fc87821c None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:13.827883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61b5ff-80a7-4420-8f84-baf2fc87821c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:13.903057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-1a61b5ff-80a7-4420-8f84-baf2fc87821c None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:13.904690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-1a61b5ff-80a7-4420-8f84-baf2fc87821c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:14.090738 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 878/1764] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:14.128897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c217795-4110-40bc-a7ea-ce21df7f49cb admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:14.131017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7c217795-4110-40bc-a7ea-ce21df7f49cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:14.268990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:14.366678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c217795-4110-40bc-a7ea-ce21df7f49cb admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:14.380739 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f2642-43b9-4330-b28b-e3a30da2e544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:14.382178 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f2642-43b9-4330-b28b-e3a30da2e544 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.382708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f2642-43b9-4330-b28b-e3a30da2e544 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.383129 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f2642-43b9-4330-b28b-e3a30da2e544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.413133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f2642-43b9-4330-b28b-e3a30da2e544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:14.415190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f2642-43b9-4330-b28b-e3a30da2e544 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:14.428838 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 887/1765] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:14.808946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7594ff-1aaf-4c26-a8f4-1072a83336b4 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.810088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7594ff-1aaf-4c26-a8f4-1072a83336b4 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.810671 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7594ff-1aaf-4c26-a8f4-1072a83336b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.849274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-0d7594ff-1aaf-4c26-a8f4-1072a83336b4 None None] MFA Rules not processed for user `b15f8d66d5104aaeae51e6c1fa3b3596`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:14.851858 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0d7594ff-1aaf-4c26-a8f4-1072a83336b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:15.096839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 879/1766] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:16.096218 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f2642-43b9-4330-b28b-e3a30da2e544 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:16.098118 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fc9f2642-43b9-4330-b28b-e3a30da2e544 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:16.119333 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f2642-43b9-4330-b28b-e3a30da2e544 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:16.142223 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573b4c3-8194-4b5d-bc07-cb266f57f92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:16.143375 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573b4c3-8194-4b5d-bc07-cb266f57f92b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:16.143807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573b4c3-8194-4b5d-bc07-cb266f57f92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:16.144490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573b4c3-8194-4b5d-bc07-cb266f57f92b admin admin] PATH_INFO: `/v3/projects/a72b7b54631842e4b2777eacf0a59fe8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:16.146600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573b4c3-8194-4b5d-bc07-cb266f57f92b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a72b7b54631842e4b2777eacf0a59fe8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:16.150669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573b4c3-8194-4b5d-bc07-cb266f57f92b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:16.283316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0573b4c3-8194-4b5d-bc07-cb266f57f92b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a72b7b54631842e4b2777eacf0a59fe8', 'request_id': 'req-0573b4c3-8194-4b5d-bc07-cb266f57f92b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.296609 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0573b4c3-8194-4b5d-bc07-cb266f57f92b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a72b7b54631842e4b2777eacf0a59fe8', 'request_id': 'req-0573b4c3-8194-4b5d-bc07-cb266f57f92b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.321265 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0573b4c3-8194-4b5d-bc07-cb266f57f92b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a72b7b54631842e4b2777eacf0a59fe8', 'request_id': 'req-0573b4c3-8194-4b5d-bc07-cb266f57f92b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.357133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0573b4c3-8194-4b5d-bc07-cb266f57f92b admin admin] The token cache is being invalidate because project a72b7b54631842e4b2777eacf0a59fe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:16.361109 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0573b4c3-8194-4b5d-bc07-cb266f57f92b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.367988 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0573b4c3-8194-4b5d-bc07-cb266f57f92b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.370363 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 888/1767] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:16 2022] DELETE /identity/v3/projects/a72b7b54631842e4b2777eacf0a59fe8 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:16.374307 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7594ff-1aaf-4c26-a8f4-1072a83336b4 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:16.390633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7594ff-1aaf-4c26-a8f4-1072a83336b4 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:16.484364 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e03af0-124f-4506-a146-730d7c492282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:16.486228 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e03af0-124f-4506-a146-730d7c492282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:16.486580 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e03af0-124f-4506-a146-730d7c492282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:16.486967 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e03af0-124f-4506-a146-730d7c492282 admin admin] PATH_INFO: `/v3/users/908b5c7a110e43e89156ebca1ed54ed8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:16.490145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e03af0-124f-4506-a146-730d7c492282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=908b5c7a110e43e89156ebca1ed54ed8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:16.492964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e03af0-124f-4506-a146-730d7c492282 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:16.499359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:16.539995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573b4c3-8194-4b5d-bc07-cb266f57f92b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:16.541654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0573b4c3-8194-4b5d-bc07-cb266f57f92b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:16.586200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573b4c3-8194-4b5d-bc07-cb266f57f92b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:16.600160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dce83-f181-4eae-ae96-92afff5b4201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:16.616414 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dce83-f181-4eae-ae96-92afff5b4201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:16.616717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dce83-f181-4eae-ae96-92afff5b4201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:16.616983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dce83-f181-4eae-ae96-92afff5b4201 admin admin] PATH_INFO: `/v3/projects/b93ce1e7a02a4f8bb16d03d8c32c17bb` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:16.625942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647dce83-f181-4eae-ae96-92afff5b4201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b93ce1e7a02a4f8bb16d03d8c32c17bb)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:16.628044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647dce83-f181-4eae-ae96-92afff5b4201 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:16.664983 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b5e03af0-124f-4506-a146-730d7c492282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '908b5c7a110e43e89156ebca1ed54ed8'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.704478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b5e03af0-124f-4506-a146-730d7c492282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '908b5c7a110e43e89156ebca1ed54ed8'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.726636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b5e03af0-124f-4506-a146-730d7c492282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '908b5c7a110e43e89156ebca1ed54ed8'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.737812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b5e03af0-124f-4506-a146-730d7c492282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '908b5c7a110e43e89156ebca1ed54ed8'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.740148 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b5e03af0-124f-4506-a146-730d7c492282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '908b5c7a110e43e89156ebca1ed54ed8'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.759039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 880/1768] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:16 2022] DELETE /identity/v3/users/908b5c7a110e43e89156ebca1ed54ed8 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:16.782299 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a0463-b179-4316-8505-6e6cfb42d558 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:16.782600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a0463-b179-4316-8505-6e6cfb42d558 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:16.782886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a0463-b179-4316-8505-6e6cfb42d558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:16.819691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-647dce83-f181-4eae-ae96-92afff5b4201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b93ce1e7a02a4f8bb16d03d8c32c17bb', 'request_id': 'req-647dce83-f181-4eae-ae96-92afff5b4201'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.845521 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-647dce83-f181-4eae-ae96-92afff5b4201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b93ce1e7a02a4f8bb16d03d8c32c17bb', 'request_id': 'req-647dce83-f181-4eae-ae96-92afff5b4201'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.849491 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-772a0463-b179-4316-8505-6e6cfb42d558 None None] MFA Rules not processed for user `9ac790383bdf4eecaef96cc25fe38553`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:16.853001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-772a0463-b179-4316-8505-6e6cfb42d558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:16.872564 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-647dce83-f181-4eae-ae96-92afff5b4201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b93ce1e7a02a4f8bb16d03d8c32c17bb', 'request_id': 'req-647dce83-f181-4eae-ae96-92afff5b4201'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.893210 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-647dce83-f181-4eae-ae96-92afff5b4201 admin admin] The token cache is being invalidate because project b93ce1e7a02a4f8bb16d03d8c32c17bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:16.896015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-647dce83-f181-4eae-ae96-92afff5b4201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.897566 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-647dce83-f181-4eae-ae96-92afff5b4201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.902990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 889/1769] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:16 2022] DELETE /identity/v3/projects/b93ce1e7a02a4f8bb16d03d8c32c17bb => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:16.907094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dce83-f181-4eae-ae96-92afff5b4201 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:16.914694 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-647dce83-f181-4eae-ae96-92afff5b4201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:16.993280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:17.005683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 881/1770] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:17.050967 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dce83-f181-4eae-ae96-92afff5b4201 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:17.056852 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffca95c-2aff-4415-a075-9da082b9fd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:17.057757 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffca95c-2aff-4415-a075-9da082b9fd20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.064262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffca95c-2aff-4415-a075-9da082b9fd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.064473 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffca95c-2aff-4415-a075-9da082b9fd20 admin admin] PATH_INFO: `/v3/users/4371317787784e91b34df08a6b964ffd` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:17.066346 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffca95c-2aff-4415-a075-9da082b9fd20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4371317787784e91b34df08a6b964ffd)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:17.068678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffca95c-2aff-4415-a075-9da082b9fd20 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:17.160579 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2ffca95c-2aff-4415-a075-9da082b9fd20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4371317787784e91b34df08a6b964ffd'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:17.173683 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2ffca95c-2aff-4415-a075-9da082b9fd20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4371317787784e91b34df08a6b964ffd'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:17.174922 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2ffca95c-2aff-4415-a075-9da082b9fd20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4371317787784e91b34df08a6b964ffd'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:17.195126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2ffca95c-2aff-4415-a075-9da082b9fd20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4371317787784e91b34df08a6b964ffd'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:17.210692 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2ffca95c-2aff-4415-a075-9da082b9fd20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4371317787784e91b34df08a6b964ffd'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:17.220342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 890/1771] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:16 2022] DELETE /identity/v3/users/4371317787784e91b34df08a6b964ffd => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:17.649888 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce1dc3-488f-415b-84fc-424e1c076831 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.650964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce1dc3-488f-415b-84fc-424e1c076831 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.651445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce1dc3-488f-415b-84fc-424e1c076831 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:17.655138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 882/1772] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jul 20 19:24:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:24:17.974312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ceda4e-0e7e-4398-a81b-0dfc4ce1640f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.974605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ceda4e-0e7e-4398-a81b-0dfc4ce1640f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.974885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ceda4e-0e7e-4398-a81b-0dfc4ce1640f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:18.032534 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c1ceda4e-0e7e-4398-a81b-0dfc4ce1640f None None] MFA Rules not processed for user `267b178236e046e4bc694b2da6f86eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:18.034077 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c1ceda4e-0e7e-4398-a81b-0dfc4ce1640f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:18.182154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce1dc3-488f-415b-84fc-424e1c076831 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:18.185197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-95ce1dc3-488f-415b-84fc-424e1c076831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:18.283055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 891/1773] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jul 20 19:24:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:18.420073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce1dc3-488f-415b-84fc-424e1c076831 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:18.443101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab764b-0b28-4821-be0c-0466d30151bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:18.443870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab764b-0b28-4821-be0c-0466d30151bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:18.444546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab764b-0b28-4821-be0c-0466d30151bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:18.444546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab764b-0b28-4821-be0c-0466d30151bd admin admin] PATH_INFO: `/v3/users/0be76125f91745bb8d6d723ac8adfae2` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:18.446235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab764b-0b28-4821-be0c-0466d30151bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0be76125f91745bb8d6d723ac8adfae2)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:18.448244 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab764b-0b28-4821-be0c-0466d30151bd admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:18.520866 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:18.555964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-beab764b-0b28-4821-be0c-0466d30151bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0be76125f91745bb8d6d723ac8adfae2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:18.574344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-beab764b-0b28-4821-be0c-0466d30151bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0be76125f91745bb8d6d723ac8adfae2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:18.596679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-beab764b-0b28-4821-be0c-0466d30151bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0be76125f91745bb8d6d723ac8adfae2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:18.618464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-beab764b-0b28-4821-be0c-0466d30151bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0be76125f91745bb8d6d723ac8adfae2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:18.620160 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-beab764b-0b28-4821-be0c-0466d30151bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0be76125f91745bb8d6d723ac8adfae2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:18.631614 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 883/1774] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:18 2022] DELETE /identity/v3/users/0be76125f91745bb8d6d723ac8adfae2 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:18.637680 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc322382-a8ed-4109-85ba-725d6659850b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:18.638301 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc322382-a8ed-4109-85ba-725d6659850b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:18.638679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc322382-a8ed-4109-85ba-725d6659850b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:18.681567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-bc322382-a8ed-4109-85ba-725d6659850b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:18.684966 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bc322382-a8ed-4109-85ba-725d6659850b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:18.713642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab764b-0b28-4821-be0c-0466d30151bd admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:18.715992 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-beab764b-0b28-4821-be0c-0466d30151bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:18.877934 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab764b-0b28-4821-be0c-0466d30151bd admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:18.892101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85948b78-95e7-4d24-bf29-b1166fe3092f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:18.893160 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85948b78-95e7-4d24-bf29-b1166fe3092f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:18.895155 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85948b78-95e7-4d24-bf29-b1166fe3092f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:18.896455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85948b78-95e7-4d24-bf29-b1166fe3092f admin admin] PATH_INFO: `/v3/projects/f537f6c99e3b495a829fe6f82150c8d0` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:18.898909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85948b78-95e7-4d24-bf29-b1166fe3092f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f537f6c99e3b495a829fe6f82150c8d0)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:18.904381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85948b78-95e7-4d24-bf29-b1166fe3092f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:18.969073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:19.072832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 892/1775] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:19.123208 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc322382-a8ed-4109-85ba-725d6659850b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.126323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bc322382-a8ed-4109-85ba-725d6659850b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:19.170956 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-85948b78-95e7-4d24-bf29-b1166fe3092f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f537f6c99e3b495a829fe6f82150c8d0', 'request_id': 'req-85948b78-95e7-4d24-bf29-b1166fe3092f'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:19.204115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-85948b78-95e7-4d24-bf29-b1166fe3092f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f537f6c99e3b495a829fe6f82150c8d0', 'request_id': 'req-85948b78-95e7-4d24-bf29-b1166fe3092f'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:19.239111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-85948b78-95e7-4d24-bf29-b1166fe3092f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f537f6c99e3b495a829fe6f82150c8d0', 'request_id': 'req-85948b78-95e7-4d24-bf29-b1166fe3092f'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:19.275077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-85948b78-95e7-4d24-bf29-b1166fe3092f admin admin] The token cache is being invalidate because project f537f6c99e3b495a829fe6f82150c8d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:19.275077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-85948b78-95e7-4d24-bf29-b1166fe3092f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:19.276660 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-85948b78-95e7-4d24-bf29-b1166fe3092f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:19.279572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc322382-a8ed-4109-85ba-725d6659850b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.280002 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 884/1776] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:18 2022] DELETE /identity/v3/projects/f537f6c99e3b495a829fe6f82150c8d0 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:19.286707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cb6f7-c649-42f4-9e3e-87f327008ae0 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.296712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cb6f7-c649-42f4-9e3e-87f327008ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.297179 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cb6f7-c649-42f4-9e3e-87f327008ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.356698 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-e98cb6f7-c649-42f4-9e3e-87f327008ae0 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:19.368496 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e98cb6f7-c649-42f4-9e3e-87f327008ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:19.392001 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec17499-5fe5-4a7c-83d5-5d84073d6860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.392771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec17499-5fe5-4a7c-83d5-5d84073d6860 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.393065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec17499-5fe5-4a7c-83d5-5d84073d6860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.394800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec17499-5fe5-4a7c-83d5-5d84073d6860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.402959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4ec17499-5fe5-4a7c-83d5-5d84073d6860 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:19.494915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:19.560815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec17499-5fe5-4a7c-83d5-5d84073d6860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.561977 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec17499-5fe5-4a7c-83d5-5d84073d6860 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.579190 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 885/1777] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:19.580267 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98cb6f7-c649-42f4-9e3e-87f327008ae0 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.585223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e98cb6f7-c649-42f4-9e3e-87f327008ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:19.598436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 893/1778] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:19.605750 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3b894-20af-4e02-ac4b-d4d67458d455 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.606838 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3b894-20af-4e02-ac4b-d4d67458d455 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.606838 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3b894-20af-4e02-ac4b-d4d67458d455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.665753 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-62f3b894-20af-4e02-ac4b-d4d67458d455 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:19.669728 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-62f3b894-20af-4e02-ac4b-d4d67458d455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:19.693723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 894/1779] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:19.697544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3b894-20af-4e02-ac4b-d4d67458d455 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.719382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3b894-20af-4e02-ac4b-d4d67458d455 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.733036 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa340005-70fa-4f14-bb05-62c1eadf8718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.736678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa340005-70fa-4f14-bb05-62c1eadf8718 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.736989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa340005-70fa-4f14-bb05-62c1eadf8718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.737340 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa340005-70fa-4f14-bb05-62c1eadf8718 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.740974 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa340005-70fa-4f14-bb05-62c1eadf8718 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:19.742193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa340005-70fa-4f14-bb05-62c1eadf8718 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.745292 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa340005-70fa-4f14-bb05-62c1eadf8718 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.784676 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 895/1780] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:24:19.786019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa340005-70fa-4f14-bb05-62c1eadf8718 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.803432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98cb6f7-c649-42f4-9e3e-87f327008ae0 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.812149 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa340005-70fa-4f14-bb05-62c1eadf8718 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.819418 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf8afa-f5e3-48bc-bb3a-28d161b89ec1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.821424 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf8afa-f5e3-48bc-bb3a-28d161b89ec1 service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.821725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf8afa-f5e3-48bc-bb3a-28d161b89ec1 service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.822008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf8afa-f5e3-48bc-bb3a-28d161b89ec1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.828829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-97cf8afa-f5e3-48bc-bb3a-28d161b89ec1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:19.829601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc422d-6c6c-4c4e-98d6-0a2d54876fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.831146 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc422d-6c6c-4c4e-98d6-0a2d54876fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.831434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc422d-6c6c-4c4e-98d6-0a2d54876fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.831711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc422d-6c6c-4c4e-98d6-0a2d54876fe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.832456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc422d-6c6c-4c4e-98d6-0a2d54876fe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.841268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc422d-6c6c-4c4e-98d6-0a2d54876fe2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.869866 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf8afa-f5e3-48bc-bb3a-28d161b89ec1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.871495 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf8afa-f5e3-48bc-bb3a-28d161b89ec1 service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.912215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 896/1781] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:19 2022] POST /identity/v3/projects => generated 365 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:19.912461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 886/1782] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:19.923571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc422d-6c6c-4c4e-98d6-0a2d54876fe2 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.959702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:20.173432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc422d-6c6c-4c4e-98d6-0a2d54876fe2 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:20.188993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe95e14-d74f-4163-9ce0-980ff2b23bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:20.196385 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe95e14-d74f-4163-9ce0-980ff2b23bae admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:20.196697 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe95e14-d74f-4163-9ce0-980ff2b23bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:20.196927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe95e14-d74f-4163-9ce0-980ff2b23bae admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:20.198638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe95e14-d74f-4163-9ce0-980ff2b23bae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:20.202290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe95e14-d74f-4163-9ce0-980ff2b23bae admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:20.244034 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf8afa-f5e3-48bc-bb3a-28d161b89ec1 service nova] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:20.247388 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-97cf8afa-f5e3-48bc-bb3a-28d161b89ec1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:20.274308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 897/1783] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:19 2022] POST /identity/v3/users => generated 283 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:20.274981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe95e14-d74f-4163-9ce0-980ff2b23bae admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:20.287887 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf8afa-f5e3-48bc-bb3a-28d161b89ec1 service nova] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:20.313915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69156394-6521-41eb-85e7-0892060e00c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:20.314713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69156394-6521-41eb-85e7-0892060e00c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:20.314995 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69156394-6521-41eb-85e7-0892060e00c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:20.315266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69156394-6521-41eb-85e7-0892060e00c2 admin admin] PATH_INFO: `/v3/projects/09ceaa7b34b149b3a62be8401e194981` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:20.321241 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69156394-6521-41eb-85e7-0892060e00c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09ceaa7b34b149b3a62be8401e194981)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:20.323108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69156394-6521-41eb-85e7-0892060e00c2 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:20.325347 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe95e14-d74f-4163-9ce0-980ff2b23bae admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:20.356984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4cc9e-3277-4ac0-bc6c-1f3a42521299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:20.357674 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4cc9e-3277-4ac0-bc6c-1f3a42521299 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:20.357956 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4cc9e-3277-4ac0-bc6c-1f3a42521299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:20.360043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4cc9e-3277-4ac0-bc6c-1f3a42521299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:20.360043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4cc9e-3277-4ac0-bc6c-1f3a42521299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:20.360602 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4cc9e-3277-4ac0-bc6c-1f3a42521299 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:20.419525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 898/1784] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:20.428097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4cc9e-3277-4ac0-bc6c-1f3a42521299 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:20.478894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4cc9e-3277-4ac0-bc6c-1f3a42521299 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:20.500165 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21a730-2bd4-42ed-8dd8-ba91144d75bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:20.501061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21a730-2bd4-42ed-8dd8-ba91144d75bc admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:20.501161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21a730-2bd4-42ed-8dd8-ba91144d75bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:20.501426 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21a730-2bd4-42ed-8dd8-ba91144d75bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:20.502162 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21a730-2bd4-42ed-8dd8-ba91144d75bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:20.516131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21a730-2bd4-42ed-8dd8-ba91144d75bc admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:20.561920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-69156394-6521-41eb-85e7-0892060e00c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09ceaa7b34b149b3a62be8401e194981', 'request_id': 'req-69156394-6521-41eb-85e7-0892060e00c2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:20.579726 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 899/1785] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:20.583552 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21a730-2bd4-42ed-8dd8-ba91144d75bc admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:20.588420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-69156394-6521-41eb-85e7-0892060e00c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09ceaa7b34b149b3a62be8401e194981', 'request_id': 'req-69156394-6521-41eb-85e7-0892060e00c2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:20.603657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-69156394-6521-41eb-85e7-0892060e00c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09ceaa7b34b149b3a62be8401e194981', 'request_id': 'req-69156394-6521-41eb-85e7-0892060e00c2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:20.607329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21a730-2bd4-42ed-8dd8-ba91144d75bc admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:20.643408 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-69156394-6521-41eb-85e7-0892060e00c2 admin admin] The token cache is being invalidate because project 09ceaa7b34b149b3a62be8401e194981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:20.643704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-69156394-6521-41eb-85e7-0892060e00c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:20.656828 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-69156394-6521-41eb-85e7-0892060e00c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:20.667517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 887/1786] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:20 2022] DELETE /identity/v3/projects/09ceaa7b34b149b3a62be8401e194981 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:20.669920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93a98a-8dd2-4d0f-a4ba-dcaf9f0a392f None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:20.670549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93a98a-8dd2-4d0f-a4ba-dcaf9f0a392f None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:20.670549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93a98a-8dd2-4d0f-a4ba-dcaf9f0a392f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:20.715508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4d93a98a-8dd2-4d0f-a4ba-dcaf9f0a392f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:20.717274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4d93a98a-8dd2-4d0f-a4ba-dcaf9f0a392f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:20.818740 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:20.852330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 888/1787] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:20.860138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeb6b1-85fd-4919-80a4-d9db6736c43d None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:20.860765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeb6b1-85fd-4919-80a4-d9db6736c43d None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:20.861612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeb6b1-85fd-4919-80a4-d9db6736c43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:20.870590 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9e134-55c9-44a5-b81e-35996f93b99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:20.874304 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9e134-55c9-44a5-b81e-35996f93b99f admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:20.874692 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9e134-55c9-44a5-b81e-35996f93b99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:20.875177 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9e134-55c9-44a5-b81e-35996f93b99f admin admin] PATH_INFO: `/v3/projects/1e1732faff4e4e34a9483316eccdb17b/users/c0281dafd87540349b33008e50028954/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:20.900122 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-28eeb6b1-85fd-4919-80a4-d9db6736c43d None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:20.903134 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-28eeb6b1-85fd-4919-80a4-d9db6736c43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:20.919038 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 889/1788] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:20.922926 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eeb6b1-85fd-4919-80a4-d9db6736c43d None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:20.935613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9e134-55c9-44a5-b81e-35996f93b99f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e1732faff4e4e34a9483316eccdb17b, user_id=c0281dafd87540349b33008e50028954, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:20.939028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9e134-55c9-44a5-b81e-35996f93b99f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:20.951411 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eeb6b1-85fd-4919-80a4-d9db6736c43d None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.069290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 900/1789] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:20 2022] PUT /identity/v3/projects/1e1732faff4e4e34a9483316eccdb17b/users/c0281dafd87540349b33008e50028954/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:21.072734 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13862fa-c4d6-4174-a2fb-1aa51198065b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.073090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13862fa-c4d6-4174-a2fb-1aa51198065b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.073443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13862fa-c4d6-4174-a2fb-1aa51198065b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.118014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e50f7d-2eaa-4f65-a355-c91215c4f63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.119031 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e50f7d-2eaa-4f65-a355-c91215c4f63b admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.119503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e50f7d-2eaa-4f65-a355-c91215c4f63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.119962 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e50f7d-2eaa-4f65-a355-c91215c4f63b admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.120972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e50f7d-2eaa-4f65-a355-c91215c4f63b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:21.122331 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e50f7d-2eaa-4f65-a355-c91215c4f63b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:21.129478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e50f7d-2eaa-4f65-a355-c91215c4f63b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:21.155124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 890/1790] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:21.159242 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e50f7d-2eaa-4f65-a355-c91215c4f63b admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:21.179645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e13862fa-c4d6-4174-a2fb-1aa51198065b None None] MFA Rules not processed for user `c0281dafd87540349b33008e50028954`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:21.181129 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e13862fa-c4d6-4174-a2fb-1aa51198065b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:21.194151 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e50f7d-2eaa-4f65-a355-c91215c4f63b admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.204166 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c1b19-1f68-4fe2-9c09-0572df76262c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.205142 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c1b19-1f68-4fe2-9c09-0572df76262c admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.206050 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c1b19-1f68-4fe2-9c09-0572df76262c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.206495 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c1b19-1f68-4fe2-9c09-0572df76262c admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.207553 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c1b19-1f68-4fe2-9c09-0572df76262c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:21.210055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c1b19-1f68-4fe2-9c09-0572df76262c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:21.263758 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 891/1791] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:21 2022] POST /identity/v3/projects => generated 409 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:21.267289 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c1b19-1f68-4fe2-9c09-0572df76262c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:21.275156 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:21.316084 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 901/1792] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:21.322614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13862fa-c4d6-4174-a2fb-1aa51198065b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:21.324789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e13862fa-c4d6-4174-a2fb-1aa51198065b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:21.374941 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13862fa-c4d6-4174-a2fb-1aa51198065b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.384277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c1b19-1f68-4fe2-9c09-0572df76262c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.388556 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c22b5-7d12-489a-9f75-04f394677536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.389260 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c22b5-7d12-489a-9f75-04f394677536 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.389690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c22b5-7d12-489a-9f75-04f394677536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.390206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c22b5-7d12-489a-9f75-04f394677536 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.391550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c22b5-7d12-489a-9f75-04f394677536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:21.399041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c22b5-7d12-489a-9f75-04f394677536 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:21.402415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32e405-f92d-4fe7-8e54-221c95ede938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.403464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32e405-f92d-4fe7-8e54-221c95ede938 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.403755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32e405-f92d-4fe7-8e54-221c95ede938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.404036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32e405-f92d-4fe7-8e54-221c95ede938 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.404784 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32e405-f92d-4fe7-8e54-221c95ede938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:21.407063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32e405-f92d-4fe7-8e54-221c95ede938 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:21.451963 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 892/1793] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:21 2022] POST /identity/v3/users => generated 305 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:21.458916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32e405-f92d-4fe7-8e54-221c95ede938 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:21.460130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 902/1794] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:21 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:21.467531 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c22b5-7d12-489a-9f75-04f394677536 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:21.489990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32e405-f92d-4fe7-8e54-221c95ede938 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.554223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:21.596561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a67c9-412d-45d9-a99c-065adb308125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.597340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a67c9-412d-45d9-a99c-065adb308125 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.598184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a67c9-412d-45d9-a99c-065adb308125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.599174 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a67c9-412d-45d9-a99c-065adb308125 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.600184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a67c9-412d-45d9-a99c-065adb308125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:21.602148 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a67c9-412d-45d9-a99c-065adb308125 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:21.618585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 893/1795] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:21.618585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a372b-ac91-4eeb-96ac-0399a572ab37 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.618585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a372b-ac91-4eeb-96ac-0399a572ab37 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.618585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a372b-ac91-4eeb-96ac-0399a572ab37 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.621067 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 894/1796] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jul 20 19:24:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:24:21.622032 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a372b-ac91-4eeb-96ac-0399a572ab37 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:21.623911 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5b4a372b-ac91-4eeb-96ac-0399a572ab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:21.638008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c22b5-7d12-489a-9f75-04f394677536 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.644071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a02110-0224-49ce-bc8c-06546f9880e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.646112 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a372b-ac91-4eeb-96ac-0399a572ab37 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.647072 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a02110-0224-49ce-bc8c-06546f9880e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.647443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a02110-0224-49ce-bc8c-06546f9880e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.647823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a02110-0224-49ce-bc8c-06546f9880e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.649041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a02110-0224-49ce-bc8c-06546f9880e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:21.652630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a02110-0224-49ce-bc8c-06546f9880e8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:21.668471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195f68a-dd8a-494e-915a-b0feac0d11c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.669917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195f68a-dd8a-494e-915a-b0feac0d11c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.670356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195f68a-dd8a-494e-915a-b0feac0d11c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.670793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195f68a-dd8a-494e-915a-b0feac0d11c5 admin admin] PATH_INFO: `/v3/projects/fb22045070644bfc9a3574ad6b004a81` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.677555 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195f68a-dd8a-494e-915a-b0feac0d11c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb22045070644bfc9a3574ad6b004a81)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:21.682717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195f68a-dd8a-494e-915a-b0feac0d11c5 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:21.691167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 903/1797] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:21 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:21.692808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a02110-0224-49ce-bc8c-06546f9880e8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:21.705866 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a02110-0224-49ce-bc8c-06546f9880e8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.714424 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655ee62-ee0a-464c-bdac-cfac2611ba77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.715196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655ee62-ee0a-464c-bdac-cfac2611ba77 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.715486 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655ee62-ee0a-464c-bdac-cfac2611ba77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.715810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655ee62-ee0a-464c-bdac-cfac2611ba77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.716594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655ee62-ee0a-464c-bdac-cfac2611ba77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:21.718260 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655ee62-ee0a-464c-bdac-cfac2611ba77 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:21.745480 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 904/1798] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:21.747731 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3ff9d-fd0a-4133-ac18-e90dd12bf48c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.748187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3ff9d-fd0a-4133-ac18-e90dd12bf48c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.748658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3ff9d-fd0a-4133-ac18-e90dd12bf48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.768125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b195f68a-dd8a-494e-915a-b0feac0d11c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb22045070644bfc9a3574ad6b004a81', 'request_id': 'req-b195f68a-dd8a-494e-915a-b0feac0d11c5'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:21.775027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0ac3ff9d-fd0a-4133-ac18-e90dd12bf48c None None] MFA Rules not processed for user `267b178236e046e4bc694b2da6f86eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:21.776276 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0ac3ff9d-fd0a-4133-ac18-e90dd12bf48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:21.782109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b195f68a-dd8a-494e-915a-b0feac0d11c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb22045070644bfc9a3574ad6b004a81', 'request_id': 'req-b195f68a-dd8a-494e-915a-b0feac0d11c5'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:21.789679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b195f68a-dd8a-494e-915a-b0feac0d11c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb22045070644bfc9a3574ad6b004a81', 'request_id': 'req-b195f68a-dd8a-494e-915a-b0feac0d11c5'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:21.804124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b195f68a-dd8a-494e-915a-b0feac0d11c5 admin admin] The token cache is being invalidate because project fb22045070644bfc9a3574ad6b004a81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:21.804807 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b195f68a-dd8a-494e-915a-b0feac0d11c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:21.805342 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b195f68a-dd8a-494e-915a-b0feac0d11c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:21.807035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 895/1799] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:21 2022] DELETE /identity/v3/projects/fb22045070644bfc9a3574ad6b004a81 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:21.807842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195f68a-dd8a-494e-915a-b0feac0d11c5 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:21.810750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-b195f68a-dd8a-494e-915a-b0feac0d11c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:21.894385 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 905/1800] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jul 20 19:24:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:21.897876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3ff9d-fd0a-4133-ac18-e90dd12bf48c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:21.899618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0ac3ff9d-fd0a-4133-ac18-e90dd12bf48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:21.919102 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195f68a-dd8a-494e-915a-b0feac0d11c5 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.924728 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdff11c-0866-41ed-975b-8574a2688448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.925409 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdff11c-0866-41ed-975b-8574a2688448 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.925696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdff11c-0866-41ed-975b-8574a2688448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.926286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdff11c-0866-41ed-975b-8574a2688448 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.927295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdff11c-0866-41ed-975b-8574a2688448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:21.928942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdff11c-0866-41ed-975b-8574a2688448 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:21.944710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 896/1801] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:21.945678 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdff11c-0866-41ed-975b-8574a2688448 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:21.947365 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-dfdff11c-0866-41ed-975b-8574a2688448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:21.957056 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:21.962359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdff11c-0866-41ed-975b-8574a2688448 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.967808 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b3e53-ab2a-4657-9595-7def96dfe982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.968726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b3e53-ab2a-4657-9595-7def96dfe982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.969007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b3e53-ab2a-4657-9595-7def96dfe982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.969282 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b3e53-ab2a-4657-9595-7def96dfe982 admin admin] PATH_INFO: `/v3/users/596600e15d2649d9b47ed82ce7633d4d` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.970576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b3e53-ab2a-4657-9595-7def96dfe982 admin admin] RBAC: Authorizing `identity:delete_user(user_id=596600e15d2649d9b47ed82ce7633d4d)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:21.972529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b3e53-ab2a-4657-9595-7def96dfe982 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:21.991153 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3ff9d-fd0a-4133-ac18-e90dd12bf48c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.995730 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c558d-d413-4b1d-9f1d-b4209b2ecd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.996643 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c558d-d413-4b1d-9f1d-b4209b2ecd08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.996929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c558d-d413-4b1d-9f1d-b4209b2ecd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.997200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c558d-d413-4b1d-9f1d-b4209b2ecd08 admin admin] PATH_INFO: `/v3/projects/01cf8ffff9b3457f8c99672b545e5dc8/users/2366633e5c6748f9be5c48d89c0c450e/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.014714 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c558d-d413-4b1d-9f1d-b4209b2ecd08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01cf8ffff9b3457f8c99672b545e5dc8, user_id=2366633e5c6748f9be5c48d89c0c450e, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.017404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c558d-d413-4b1d-9f1d-b4209b2ecd08 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.026721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a30b3e53-ab2a-4657-9595-7def96dfe982 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '596600e15d2649d9b47ed82ce7633d4d'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.030509 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:22.040366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 906/1802] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:21 2022] PUT /identity/v3/projects/01cf8ffff9b3457f8c99672b545e5dc8/users/2366633e5c6748f9be5c48d89c0c450e/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:22.040366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c558d-d413-4b1d-9f1d-b4209b2ecd08 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.040366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-774c558d-d413-4b1d-9f1d-b4209b2ecd08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:22.044233 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a30b3e53-ab2a-4657-9595-7def96dfe982 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '596600e15d2649d9b47ed82ce7633d4d'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.059789 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a30b3e53-ab2a-4657-9595-7def96dfe982 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '596600e15d2649d9b47ed82ce7633d4d'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.068977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a30b3e53-ab2a-4657-9595-7def96dfe982 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '596600e15d2649d9b47ed82ce7633d4d'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.069720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a30b3e53-ab2a-4657-9595-7def96dfe982 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '596600e15d2649d9b47ed82ce7633d4d'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.078289 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 897/1803] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:21 2022] DELETE /identity/v3/users/596600e15d2649d9b47ed82ce7633d4d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:22.079062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b3e53-ab2a-4657-9595-7def96dfe982 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.081144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-a30b3e53-ab2a-4657-9595-7def96dfe982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:22.148059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c558d-d413-4b1d-9f1d-b4209b2ecd08 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.178804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b3e53-ab2a-4657-9595-7def96dfe982 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.185288 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d412a-84a6-4399-b285-8b8f50d74a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.186892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d412a-84a6-4399-b285-8b8f50d74a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.187261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d412a-84a6-4399-b285-8b8f50d74a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.187723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d412a-84a6-4399-b285-8b8f50d74a66 admin admin] PATH_INFO: `/v3/projects/e4d529e97c914403a74cf39ecda22201/users/16bf7a4c80e24970a80e1d8dc45ec301/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.214043 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d412a-84a6-4399-b285-8b8f50d74a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d529e97c914403a74cf39ecda22201, user_id=16bf7a4c80e24970a80e1d8dc45ec301, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.217409 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d412a-84a6-4399-b285-8b8f50d74a66 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.246441 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 898/1804] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:22 2022] PUT /identity/v3/projects/e4d529e97c914403a74cf39ecda22201/users/16bf7a4c80e24970a80e1d8dc45ec301/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:22.248365 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36059be9-8acd-44ee-80aa-e8d9306a0993 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.248724 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36059be9-8acd-44ee-80aa-e8d9306a0993 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.249046 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36059be9-8acd-44ee-80aa-e8d9306a0993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.249732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ec51f-46d0-4b1e-8bcc-2d89ae65f736 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.250406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ec51f-46d0-4b1e-8bcc-2d89ae65f736 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.250691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ec51f-46d0-4b1e-8bcc-2d89ae65f736 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.250969 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ec51f-46d0-4b1e-8bcc-2d89ae65f736 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.253721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-252ec51f-46d0-4b1e-8bcc-2d89ae65f736 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:22.275370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ec51f-46d0-4b1e-8bcc-2d89ae65f736 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.277199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ec51f-46d0-4b1e-8bcc-2d89ae65f736 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.289418 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 907/1805] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:22.290155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ec51f-46d0-4b1e-8bcc-2d89ae65f736 service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.302796 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-36059be9-8acd-44ee-80aa-e8d9306a0993 None None] MFA Rules not processed for user `2366633e5c6748f9be5c48d89c0c450e`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:22.304531 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ec51f-46d0-4b1e-8bcc-2d89ae65f736 service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.304911 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-36059be9-8acd-44ee-80aa-e8d9306a0993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:22.370100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:22.388216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 899/1806] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:22 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:22.388820 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb7a77-cef1-4f57-9c92-51e260534622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.389279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36059be9-8acd-44ee-80aa-e8d9306a0993 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.389555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb7a77-cef1-4f57-9c92-51e260534622 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.389944 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb7a77-cef1-4f57-9c92-51e260534622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.390289 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb7a77-cef1-4f57-9c92-51e260534622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.391031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb7a77-cef1-4f57-9c92-51e260534622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.391348 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-36059be9-8acd-44ee-80aa-e8d9306a0993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:22.393485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb7a77-cef1-4f57-9c92-51e260534622 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.404742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 908/1807] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:22.408571 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36059be9-8acd-44ee-80aa-e8d9306a0993 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.408939 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb7a77-cef1-4f57-9c92-51e260534622 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.414011 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4944ba-3069-4ac8-b062-bb712bce6c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.414968 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4944ba-3069-4ac8-b062-bb712bce6c38 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.415257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4944ba-3069-4ac8-b062-bb712bce6c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.416084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4944ba-3069-4ac8-b062-bb712bce6c38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.416957 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4944ba-3069-4ac8-b062-bb712bce6c38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.418879 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4944ba-3069-4ac8-b062-bb712bce6c38 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.419308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb7a77-cef1-4f57-9c92-51e260534622 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.425705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9150a-6eb9-4606-b7a0-6922a8d29d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.427276 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9150a-6eb9-4606-b7a0-6922a8d29d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.427597 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9150a-6eb9-4606-b7a0-6922a8d29d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.427876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9150a-6eb9-4606-b7a0-6922a8d29d28 admin admin] PATH_INFO: `/v3/projects/e4d529e97c914403a74cf39ecda22201/users/16bf7a4c80e24970a80e1d8dc45ec301/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.433963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9150a-6eb9-4606-b7a0-6922a8d29d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d529e97c914403a74cf39ecda22201, user_id=16bf7a4c80e24970a80e1d8dc45ec301, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.437599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9150a-6eb9-4606-b7a0-6922a8d29d28 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.452941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 900/1808] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:22 2022] POST /identity/v3/projects => generated 409 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:22.454308 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4944ba-3069-4ac8-b062-bb712bce6c38 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.455721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 909/1809] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:22 2022] PUT /identity/v3/projects/e4d529e97c914403a74cf39ecda22201/users/16bf7a4c80e24970a80e1d8dc45ec301/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:22.456470 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-9d4944ba-3069-4ac8-b062-bb712bce6c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:22.457146 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9150a-6eb9-4606-b7a0-6922a8d29d28 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.479822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:22.524254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4944ba-3069-4ac8-b062-bb712bce6c38 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.527725 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9150a-6eb9-4606-b7a0-6922a8d29d28 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.529143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.530094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.530450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.530727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6 admin admin] PATH_INFO: `/v3/projects/fcc0de8adfe547d999d5d45c83b682dc` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.531889 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcc0de8adfe547d999d5d45c83b682dc)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.532366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380cd65-a290-46a9-8ae3-0f2dde6815b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.533038 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380cd65-a290-46a9-8ae3-0f2dde6815b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.533324 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380cd65-a290-46a9-8ae3-0f2dde6815b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.533608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380cd65-a290-46a9-8ae3-0f2dde6815b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.533820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.534297 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380cd65-a290-46a9-8ae3-0f2dde6815b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.537119 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380cd65-a290-46a9-8ae3-0f2dde6815b7 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.559608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 910/1810] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:22 2022] POST /identity/v3/users => generated 311 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:22.561141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab63e4b-3444-4554-934d-d35757f946e0 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.562017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab63e4b-3444-4554-934d-d35757f946e0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.562155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab63e4b-3444-4554-934d-d35757f946e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.586134 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcc0de8adfe547d999d5d45c83b682dc', 'request_id': 'req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.590772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-dab63e4b-3444-4554-934d-d35757f946e0 None None] MFA Rules not processed for user `16bf7a4c80e24970a80e1d8dc45ec301`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:22.592246 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-dab63e4b-3444-4554-934d-d35757f946e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:22.596325 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcc0de8adfe547d999d5d45c83b682dc', 'request_id': 'req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.603890 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcc0de8adfe547d999d5d45c83b682dc', 'request_id': 'req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.618225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6 admin admin] The token cache is being invalidate because project fcc0de8adfe547d999d5d45c83b682dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:22.618659 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.619367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.621200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 901/1811] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:22 2022] DELETE /identity/v3/projects/fcc0de8adfe547d999d5d45c83b682dc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:22.625008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.626391 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:22.673867 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 911/1812] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:22 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:22.675174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57e947-599b-4835-a88b-83cb20e892a1 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.675464 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57e947-599b-4835-a88b-83cb20e892a1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.675747 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57e947-599b-4835-a88b-83cb20e892a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.698386 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ac57e947-599b-4835-a88b-83cb20e892a1 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:22.699670 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ac57e947-599b-4835-a88b-83cb20e892a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:22.706558 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6508631-b0a9-4cb6-9ce5-e521359bd7b6 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.710298 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 912/1813] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:22.710812 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57e947-599b-4835-a88b-83cb20e892a1 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.720791 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ac57e947-599b-4835-a88b-83cb20e892a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:22.725504 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64913bf4-a961-49fd-8c46-fb83d4eb7acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.726195 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64913bf4-a961-49fd-8c46-fb83d4eb7acf admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.726478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64913bf4-a961-49fd-8c46-fb83d4eb7acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.726752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64913bf4-a961-49fd-8c46-fb83d4eb7acf admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.727499 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64913bf4-a961-49fd-8c46-fb83d4eb7acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.729139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64913bf4-a961-49fd-8c46-fb83d4eb7acf admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.745362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 902/1814] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:22.745362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce8448-b1bf-438a-a2af-122548eca761 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.745362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce8448-b1bf-438a-a2af-122548eca761 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.745362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce8448-b1bf-438a-a2af-122548eca761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.761734 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-35ce8448-b1bf-438a-a2af-122548eca761 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:22.762938 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-35ce8448-b1bf-438a-a2af-122548eca761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:22.775523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 903/1815] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:22.775523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce8448-b1bf-438a-a2af-122548eca761 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.776914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:22.787600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce8448-b1bf-438a-a2af-122548eca761 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.793537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3828f-73cd-44fd-9d46-c862093bd8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.794107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57e947-599b-4835-a88b-83cb20e892a1 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.794436 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3828f-73cd-44fd-9d46-c862093bd8f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.794817 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3828f-73cd-44fd-9d46-c862093bd8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.795103 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3828f-73cd-44fd-9d46-c862093bd8f6 admin admin] PATH_INFO: `/v3/projects/a5a1096c346542d5ac2659b79389b462/users/964e71bacf9e43eb9939234a822675be/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.797876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d31b4-6137-4b59-99b9-ac9556f102a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.798521 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d31b4-6137-4b59-99b9-ac9556f102a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.798803 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d31b4-6137-4b59-99b9-ac9556f102a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.799074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d31b4-6137-4b59-99b9-ac9556f102a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.808678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d31b4-6137-4b59-99b9-ac9556f102a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.810112 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d31b4-6137-4b59-99b9-ac9556f102a7 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.812375 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd3828f-73cd-44fd-9d46-c862093bd8f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a1096c346542d5ac2659b79389b462, user_id=964e71bacf9e43eb9939234a822675be, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.815083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd3828f-73cd-44fd-9d46-c862093bd8f6 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.820682 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 913/1816] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:22 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:22.821572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d31b4-6137-4b59-99b9-ac9556f102a7 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.830148 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 904/1817] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:22 2022] PUT /identity/v3/projects/a5a1096c346542d5ac2659b79389b462/users/964e71bacf9e43eb9939234a822675be/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:22.834314 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d31b4-6137-4b59-99b9-ac9556f102a7 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.834668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3828f-73cd-44fd-9d46-c862093bd8f6 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.860729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:22.916623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7e5dd-06c7-454b-8935-43b876f32e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.917635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7e5dd-06c7-454b-8935-43b876f32e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.918330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7e5dd-06c7-454b-8935-43b876f32e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.918823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7e5dd-06c7-454b-8935-43b876f32e03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.919572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7e5dd-06c7-454b-8935-43b876f32e03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:22.919882 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7e5dd-06c7-454b-8935-43b876f32e03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.921506 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7e5dd-06c7-454b-8935-43b876f32e03 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.928369 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3828f-73cd-44fd-9d46-c862093bd8f6 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.933068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01186b6d-833f-45ce-84d8-e06e98dd6578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.933766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01186b6d-833f-45ce-84d8-e06e98dd6578 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.937716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 914/1818] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:22.937868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01186b6d-833f-45ce-84d8-e06e98dd6578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.937868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01186b6d-833f-45ce-84d8-e06e98dd6578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.938041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7e5dd-06c7-454b-8935-43b876f32e03 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.938129 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01186b6d-833f-45ce-84d8-e06e98dd6578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.938129 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01186b6d-833f-45ce-84d8-e06e98dd6578 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.948000 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 905/1819] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:22.949059 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01186b6d-833f-45ce-84d8-e06e98dd6578 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.959169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01186b6d-833f-45ce-84d8-e06e98dd6578 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.963947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586a711-71fe-4817-ab4d-0dff99ae564f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.964709 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586a711-71fe-4817-ab4d-0dff99ae564f admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.965000 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586a711-71fe-4817-ab4d-0dff99ae564f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.965280 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586a711-71fe-4817-ab4d-0dff99ae564f admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.965977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586a711-71fe-4817-ab4d-0dff99ae564f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.967927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586a711-71fe-4817-ab4d-0dff99ae564f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.000177 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 906/1820] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:22 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:23.000177 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586a711-71fe-4817-ab4d-0dff99ae564f admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.006539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7e5dd-06c7-454b-8935-43b876f32e03 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.077944 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52ebd2-f4b2-4321-96e0-9a3ef22e5101 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.078628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52ebd2-f4b2-4321-96e0-9a3ef22e5101 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.078943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52ebd2-f4b2-4321-96e0-9a3ef22e5101 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.079218 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52ebd2-f4b2-4321-96e0-9a3ef22e5101 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.081047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5e52ebd2-f4b2-4321-96e0-9a3ef22e5101 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:23.091487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586a711-71fe-4817-ab4d-0dff99ae564f admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.096410 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42577f94-0951-4175-8a4f-69122cdb8d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.097510 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42577f94-0951-4175-8a4f-69122cdb8d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.097793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42577f94-0951-4175-8a4f-69122cdb8d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.098079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42577f94-0951-4175-8a4f-69122cdb8d63 admin admin] PATH_INFO: `/v3/projects/a5a1096c346542d5ac2659b79389b462/users/964e71bacf9e43eb9939234a822675be/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.100729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42577f94-0951-4175-8a4f-69122cdb8d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a1096c346542d5ac2659b79389b462, user_id=964e71bacf9e43eb9939234a822675be, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.103165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42577f94-0951-4175-8a4f-69122cdb8d63 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.117477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 907/1821] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:22 2022] PUT /identity/v3/projects/a5a1096c346542d5ac2659b79389b462/users/964e71bacf9e43eb9939234a822675be/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:23.118434 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42577f94-0951-4175-8a4f-69122cdb8d63 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.135319 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:23.150254 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52ebd2-f4b2-4321-96e0-9a3ef22e5101 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.151700 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52ebd2-f4b2-4321-96e0-9a3ef22e5101 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.161375 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 915/1822] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:23.162722 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba364d-8a8c-43b4-a089-7a8de407ab77 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.163016 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba364d-8a8c-43b4-a089-7a8de407ab77 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.163290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba364d-8a8c-43b4-a089-7a8de407ab77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.194718 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-14ba364d-8a8c-43b4-a089-7a8de407ab77 None None] MFA Rules not processed for user `964e71bacf9e43eb9939234a822675be`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:23.194868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42577f94-0951-4175-8a4f-69122cdb8d63 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.196034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-14ba364d-8a8c-43b4-a089-7a8de407ab77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:23.200494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105023e-40b9-49d0-adb1-df03551af645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.201116 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105023e-40b9-49d0-adb1-df03551af645 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.201563 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105023e-40b9-49d0-adb1-df03551af645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.201958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105023e-40b9-49d0-adb1-df03551af645 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.202539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0105023e-40b9-49d0-adb1-df03551af645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.204703 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0105023e-40b9-49d0-adb1-df03551af645 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.231779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 908/1823] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:23 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:23.234619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105023e-40b9-49d0-adb1-df03551af645 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.246599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105023e-40b9-49d0-adb1-df03551af645 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.252663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92a99b-ba39-4fa2-aa76-e2968e98dde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.253372 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92a99b-ba39-4fa2-aa76-e2968e98dde3 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.253656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92a99b-ba39-4fa2-aa76-e2968e98dde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.253937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92a99b-ba39-4fa2-aa76-e2968e98dde3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.254856 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92a99b-ba39-4fa2-aa76-e2968e98dde3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.256576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92a99b-ba39-4fa2-aa76-e2968e98dde3 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.268072 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 909/1824] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:23.271867 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92a99b-ba39-4fa2-aa76-e2968e98dde3 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.281892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:23.288792 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92a99b-ba39-4fa2-aa76-e2968e98dde3 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.296157 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357cf173-ec40-416c-8ba8-a40196f854d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.297101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cf173-ec40-416c-8ba8-a40196f854d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.297579 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cf173-ec40-416c-8ba8-a40196f854d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.297926 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cf173-ec40-416c-8ba8-a40196f854d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.298840 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357cf173-ec40-416c-8ba8-a40196f854d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.300656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357cf173-ec40-416c-8ba8-a40196f854d7 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.320888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 916/1825] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:23 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:23.340394 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 910/1826] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:23.344004 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba364d-8a8c-43b4-a089-7a8de407ab77 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.347747 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357cf173-ec40-416c-8ba8-a40196f854d7 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.357862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba364d-8a8c-43b4-a089-7a8de407ab77 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.363278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574147aa-8c93-44c9-8831-dc2fc854a379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.365105 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574147aa-8c93-44c9-8831-dc2fc854a379 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.365392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574147aa-8c93-44c9-8831-dc2fc854a379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.365710 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574147aa-8c93-44c9-8831-dc2fc854a379 admin admin] PATH_INFO: `/v3/projects/1652b95123c449b7b00990484330972c/users/6e3af9cfd9dd496db158f9a3677b47b4/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.382772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574147aa-8c93-44c9-8831-dc2fc854a379 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1652b95123c449b7b00990484330972c, user_id=6e3af9cfd9dd496db158f9a3677b47b4, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.385551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574147aa-8c93-44c9-8831-dc2fc854a379 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.402103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 917/1827] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:23 2022] PUT /identity/v3/projects/1652b95123c449b7b00990484330972c/users/6e3af9cfd9dd496db158f9a3677b47b4/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:23.413018 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8a46d-24e6-406c-a911-8eb242953285 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.413306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8a46d-24e6-406c-a911-8eb242953285 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.413592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8a46d-24e6-406c-a911-8eb242953285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.443120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357cf173-ec40-416c-8ba8-a40196f854d7 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.447128 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-74a8a46d-24e6-406c-a911-8eb242953285 None None] MFA Rules not processed for user `6e3af9cfd9dd496db158f9a3677b47b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:23.449877 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-74a8a46d-24e6-406c-a911-8eb242953285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:23.498236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196b9c3-1bcd-44c8-9abd-705c17fd3597 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.498927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196b9c3-1bcd-44c8-9abd-705c17fd3597 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.499220 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196b9c3-1bcd-44c8-9abd-705c17fd3597 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.499983 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196b9c3-1bcd-44c8-9abd-705c17fd3597 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.526609 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 918/1828] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:23.538851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8a46d-24e6-406c-a911-8eb242953285 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.540345 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-74a8a46d-24e6-406c-a911-8eb242953285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:23.567859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196b9c3-1bcd-44c8-9abd-705c17fd3597 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.569340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196b9c3-1bcd-44c8-9abd-705c17fd3597 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.579531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 911/1829] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:23 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:23.583946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:23.603783 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8a46d-24e6-406c-a911-8eb242953285 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.607749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c20940-8af0-4ccb-aa63-08ca7e080b60 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3e28557896e41c893f0de2590bc96d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.608434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c20940-8af0-4ccb-aa63-08ca7e080b60 service glance] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.608720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c20940-8af0-4ccb-aa63-08ca7e080b60 service glance] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.609122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c20940-8af0-4ccb-aa63-08ca7e080b60 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.618257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c20940-8af0-4ccb-aa63-08ca7e080b60 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.619688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c20940-8af0-4ccb-aa63-08ca7e080b60 service glance] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.630413 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 919/1830] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:23.775438 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196b9c3-1bcd-44c8-9abd-705c17fd3597 service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.776866 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4196b9c3-1bcd-44c8-9abd-705c17fd3597 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:23.788554 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196b9c3-1bcd-44c8-9abd-705c17fd3597 service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.792715 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec1b25-ca9e-4d50-adf9-f2a3d0dea300 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.793370 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec1b25-ca9e-4d50-adf9-f2a3d0dea300 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.793787 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec1b25-ca9e-4d50-adf9-f2a3d0dea300 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.794068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec1b25-ca9e-4d50-adf9-f2a3d0dea300 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.795852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d2ec1b25-ca9e-4d50-adf9-f2a3d0dea300 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:23.862462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:23.865202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec1b25-ca9e-4d50-adf9-f2a3d0dea300 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.866752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec1b25-ca9e-4d50-adf9-f2a3d0dea300 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.877730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 912/1831] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:23 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:24.828731 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524c7cf-15c6-4cc9-926d-7292b5cc980a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:24.830984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524c7cf-15c6-4cc9-926d-7292b5cc980a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:24.831364 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524c7cf-15c6-4cc9-926d-7292b5cc980a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:24.867898 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a524c7cf-15c6-4cc9-926d-7292b5cc980a None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:24.869172 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a524c7cf-15c6-4cc9-926d-7292b5cc980a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:24.984593 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 920/1832] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jul 20 19:24:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:24.996281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec1b25-ca9e-4d50-adf9-f2a3d0dea300 service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:25.007952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d2ec1b25-ca9e-4d50-adf9-f2a3d0dea300 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:25.193093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec1b25-ca9e-4d50-adf9-f2a3d0dea300 service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:25.199338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc83590-8232-410c-b001-8b4afb1beedd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:25.200081 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc83590-8232-410c-b001-8b4afb1beedd service swift] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:25.200857 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc83590-8232-410c-b001-8b4afb1beedd service swift] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:25.208626 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc83590-8232-410c-b001-8b4afb1beedd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:25.227540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc83590-8232-410c-b001-8b4afb1beedd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:25.228056 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc83590-8232-410c-b001-8b4afb1beedd service swift] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:25.245465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 913/1833] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:24:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:24:34.972422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=35 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:35.241073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=34 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:36.957306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524c7cf-15c6-4cc9-926d-7292b5cc980a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:37.082106 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524c7cf-15c6-4cc9-926d-7292b5cc980a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:37.102771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9464d-9124-44c7-b0ba-5dde80e7f1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:37.104094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9464d-9124-44c7-b0ba-5dde80e7f1fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:37.104888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9464d-9124-44c7-b0ba-5dde80e7f1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:37.105666 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9464d-9124-44c7-b0ba-5dde80e7f1fe admin admin] PATH_INFO: `/v3/users/2366633e5c6748f9be5c48d89c0c450e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:37.109351 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9464d-9124-44c7-b0ba-5dde80e7f1fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=2366633e5c6748f9be5c48d89c0c450e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:37.112363 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9464d-9124-44c7-b0ba-5dde80e7f1fe admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:37.204544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d6f9464d-9124-44c7-b0ba-5dde80e7f1fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2366633e5c6748f9be5c48d89c0c450e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:37.229682 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d6f9464d-9124-44c7-b0ba-5dde80e7f1fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2366633e5c6748f9be5c48d89c0c450e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:37.231587 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d6f9464d-9124-44c7-b0ba-5dde80e7f1fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2366633e5c6748f9be5c48d89c0c450e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:37.246383 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d6f9464d-9124-44c7-b0ba-5dde80e7f1fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2366633e5c6748f9be5c48d89c0c450e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:37.264816 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d6f9464d-9124-44c7-b0ba-5dde80e7f1fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2366633e5c6748f9be5c48d89c0c450e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:37.275576 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 921/1834] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:36 2022] DELETE /identity/v3/users/2366633e5c6748f9be5c48d89c0c450e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:37.276960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f3999-62b3-41d7-965c-26e28dfffcab None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:37.277780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f3999-62b3-41d7-965c-26e28dfffcab None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:37.278256 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f3999-62b3-41d7-965c-26e28dfffcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:37.311234 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-e99f3999-62b3-41d7-965c-26e28dfffcab None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:37.312994 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e99f3999-62b3-41d7-965c-26e28dfffcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:37.453567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 914/1835] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:37.515315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9464d-9124-44c7-b0ba-5dde80e7f1fe admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:37.517777 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d6f9464d-9124-44c7-b0ba-5dde80e7f1fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:37.613200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9464d-9124-44c7-b0ba-5dde80e7f1fe admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:37.617207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc24618-74a4-4673-b973-6df53210dc8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:37.617880 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc24618-74a4-4673-b973-6df53210dc8e service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:37.618217 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc24618-74a4-4673-b973-6df53210dc8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:37.618496 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc24618-74a4-4673-b973-6df53210dc8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:37.629901 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc24618-74a4-4673-b973-6df53210dc8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:37.631951 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc24618-74a4-4673-b973-6df53210dc8e service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:37.642612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 922/1836] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:38.438160 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f3999-62b3-41d7-965c-26e28dfffcab None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:38.439699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e99f3999-62b3-41d7-965c-26e28dfffcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:38.484188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f3999-62b3-41d7-965c-26e28dfffcab None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:38.491706 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500f57e-0972-4c1c-a373-d3865150e56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:38.493007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500f57e-0972-4c1c-a373-d3865150e56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:38.493660 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500f57e-0972-4c1c-a373-d3865150e56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:38.494142 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500f57e-0972-4c1c-a373-d3865150e56d admin admin] PATH_INFO: `/v3/projects/01cf8ffff9b3457f8c99672b545e5dc8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:38.495826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500f57e-0972-4c1c-a373-d3865150e56d admin admin] RBAC: Authorizing `identity:delete_project(project_id=01cf8ffff9b3457f8c99672b545e5dc8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:38.498093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500f57e-0972-4c1c-a373-d3865150e56d admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:38.596631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-0500f57e-0972-4c1c-a373-d3865150e56d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01cf8ffff9b3457f8c99672b545e5dc8', 'request_id': 'req-0500f57e-0972-4c1c-a373-d3865150e56d'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.607661 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-0500f57e-0972-4c1c-a373-d3865150e56d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01cf8ffff9b3457f8c99672b545e5dc8', 'request_id': 'req-0500f57e-0972-4c1c-a373-d3865150e56d'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.614225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-0500f57e-0972-4c1c-a373-d3865150e56d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01cf8ffff9b3457f8c99672b545e5dc8', 'request_id': 'req-0500f57e-0972-4c1c-a373-d3865150e56d'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.628279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-0500f57e-0972-4c1c-a373-d3865150e56d admin admin] The token cache is being invalidate because project 01cf8ffff9b3457f8c99672b545e5dc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:38.628678 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-0500f57e-0972-4c1c-a373-d3865150e56d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.629409 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-0500f57e-0972-4c1c-a373-d3865150e56d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.631647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 915/1837] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:38 2022] DELETE /identity/v3/projects/01cf8ffff9b3457f8c99672b545e5dc8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:38.635552 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc24618-74a4-4673-b973-6df53210dc8e service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:38.646553 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc24618-74a4-4673-b973-6df53210dc8e service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:38.673341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:38.751009 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0841ab0-b8e8-4970-8ae2-19a6d40c6641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:38.752079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0841ab0-b8e8-4970-8ae2-19a6d40c6641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:38.752490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0841ab0-b8e8-4970-8ae2-19a6d40c6641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:38.752838 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0841ab0-b8e8-4970-8ae2-19a6d40c6641 admin admin] PATH_INFO: `/v3/users/964e71bacf9e43eb9939234a822675be` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:38.755329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0841ab0-b8e8-4970-8ae2-19a6d40c6641 admin admin] RBAC: Authorizing `identity:delete_user(user_id=964e71bacf9e43eb9939234a822675be)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:38.758032 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0841ab0-b8e8-4970-8ae2-19a6d40c6641 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:38.825690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a0841ab0-b8e8-4970-8ae2-19a6d40c6641 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '964e71bacf9e43eb9939234a822675be'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.835744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a0841ab0-b8e8-4970-8ae2-19a6d40c6641 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '964e71bacf9e43eb9939234a822675be'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.836481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a0841ab0-b8e8-4970-8ae2-19a6d40c6641 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '964e71bacf9e43eb9939234a822675be'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.847851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a0841ab0-b8e8-4970-8ae2-19a6d40c6641 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '964e71bacf9e43eb9939234a822675be'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.861893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a0841ab0-b8e8-4970-8ae2-19a6d40c6641 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '964e71bacf9e43eb9939234a822675be'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.869080 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 923/1838] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:38 2022] DELETE /identity/v3/users/964e71bacf9e43eb9939234a822675be => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:39.162683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500f57e-0972-4c1c-a373-d3865150e56d admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:39.165522 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0500f57e-0972-4c1c-a373-d3865150e56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:39.182351 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500f57e-0972-4c1c-a373-d3865150e56d admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:39.277607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30817810-9a6e-4723-a658-ceb97e18ebd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:39.278895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30817810-9a6e-4723-a658-ceb97e18ebd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:39.279293 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30817810-9a6e-4723-a658-ceb97e18ebd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:39.279682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30817810-9a6e-4723-a658-ceb97e18ebd2 admin admin] PATH_INFO: `/v3/users/8b86b9f69ddd4d88b239866662407bcc` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:39.281294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30817810-9a6e-4723-a658-ceb97e18ebd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b86b9f69ddd4d88b239866662407bcc)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:39.283857 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30817810-9a6e-4723-a658-ceb97e18ebd2 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:39.363034 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:39.392784 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-30817810-9a6e-4723-a658-ceb97e18ebd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b86b9f69ddd4d88b239866662407bcc'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.401894 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-30817810-9a6e-4723-a658-ceb97e18ebd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b86b9f69ddd4d88b239866662407bcc'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.416343 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-30817810-9a6e-4723-a658-ceb97e18ebd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b86b9f69ddd4d88b239866662407bcc'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.424040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-30817810-9a6e-4723-a658-ceb97e18ebd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b86b9f69ddd4d88b239866662407bcc'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.424811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-30817810-9a6e-4723-a658-ceb97e18ebd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b86b9f69ddd4d88b239866662407bcc'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.436133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 916/1839] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:39 2022] DELETE /identity/v3/users/8b86b9f69ddd4d88b239866662407bcc => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:39.441197 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996d042-b00b-49c7-b90f-bffc162983b8 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:39.441613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996d042-b00b-49c7-b90f-bffc162983b8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:39.442028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996d042-b00b-49c7-b90f-bffc162983b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:39.460454 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c996d042-b00b-49c7-b90f-bffc162983b8 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:39.461694 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c996d042-b00b-49c7-b90f-bffc162983b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:39.555602 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 924/1840] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:39.565407 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30817810-9a6e-4723-a658-ceb97e18ebd2 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:39.567016 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-30817810-9a6e-4723-a658-ceb97e18ebd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:39.632529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30817810-9a6e-4723-a658-ceb97e18ebd2 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:39.636421 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427d952-9e07-4cd1-8430-ddf94ee3a379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:39.637079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427d952-9e07-4cd1-8430-ddf94ee3a379 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:39.637538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427d952-9e07-4cd1-8430-ddf94ee3a379 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:39.637966 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427d952-9e07-4cd1-8430-ddf94ee3a379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:39.649699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427d952-9e07-4cd1-8430-ddf94ee3a379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:39.651504 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427d952-9e07-4cd1-8430-ddf94ee3a379 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:39.662986 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 917/1841] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:39.733971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996d042-b00b-49c7-b90f-bffc162983b8 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:39.736848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c996d042-b00b-49c7-b90f-bffc162983b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:39.755819 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996d042-b00b-49c7-b90f-bffc162983b8 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:39.761665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:39.763107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:39.764216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:39.764687 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024 admin admin] PATH_INFO: `/v3/projects/a5a1096c346542d5ac2659b79389b462` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:39.766372 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a1096c346542d5ac2659b79389b462)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:39.769690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:39.845798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:39.870547 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a1096c346542d5ac2659b79389b462', 'request_id': 'req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.881600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a1096c346542d5ac2659b79389b462', 'request_id': 'req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.931960 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a1096c346542d5ac2659b79389b462', 'request_id': 'req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.954387 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024 admin admin] The token cache is being invalidate because project a5a1096c346542d5ac2659b79389b462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:39.954794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.958597 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.969855 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca86631-381f-435e-b4e3-e8d74bad3421 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:39.970326 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca86631-381f-435e-b4e3-e8d74bad3421 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:39.970736 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca86631-381f-435e-b4e3-e8d74bad3421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:39.972648 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 925/1842] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:39 2022] DELETE /identity/v3/projects/a5a1096c346542d5ac2659b79389b462 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:39.994088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-eca86631-381f-435e-b4e3-e8d74bad3421 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:39.996432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-eca86631-381f-435e-b4e3-e8d74bad3421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:40.066406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:40.068189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:40.088100 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 918/1843] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:40.096976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eab847-f8c0-4a45-b8c9-58edcd7daba8 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.097461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eab847-f8c0-4a45-b8c9-58edcd7daba8 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.097788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eab847-f8c0-4a45-b8c9-58edcd7daba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.117938 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-55eab847-f8c0-4a45-b8c9-58edcd7daba8 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:40.119163 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-55eab847-f8c0-4a45-b8c9-58edcd7daba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:40.129203 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 919/1844] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:40.133628 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eab847-f8c0-4a45-b8c9-58edcd7daba8 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:40.147614 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eab847-f8c0-4a45-b8c9-58edcd7daba8 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:40.154197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764abf6-4e7c-4c13-9f8e-cc1d1f6e51a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:40.155008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764abf6-4e7c-4c13-9f8e-cc1d1f6e51a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.155366 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764abf6-4e7c-4c13-9f8e-cc1d1f6e51a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.155706 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764abf6-4e7c-4c13-9f8e-cc1d1f6e51a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.156621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764abf6-4e7c-4c13-9f8e-cc1d1f6e51a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:40.157021 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764abf6-4e7c-4c13-9f8e-cc1d1f6e51a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:40.158790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764abf6-4e7c-4c13-9f8e-cc1d1f6e51a9 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:40.173647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3ccad-32a7-4c73-9f2d-ba6942c13024 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:40.189893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eeca4b-9eb3-4e69-aabd-55236791d833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:40.191357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eeca4b-9eb3-4e69-aabd-55236791d833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.194895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 920/1845] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:40.196535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eeca4b-9eb3-4e69-aabd-55236791d833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.197258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eeca4b-9eb3-4e69-aabd-55236791d833 admin admin] PATH_INFO: `/v3/users/c0281dafd87540349b33008e50028954` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.198426 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764abf6-4e7c-4c13-9f8e-cc1d1f6e51a9 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:40.200167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eeca4b-9eb3-4e69-aabd-55236791d833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0281dafd87540349b33008e50028954)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:40.203786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eeca4b-9eb3-4e69-aabd-55236791d833 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:40.217411 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764abf6-4e7c-4c13-9f8e-cc1d1f6e51a9 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:40.224456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138faf5a-8a74-4080-b8f1-206c01aafd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:40.225363 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138faf5a-8a74-4080-b8f1-206c01aafd03 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.225728 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138faf5a-8a74-4080-b8f1-206c01aafd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.226128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138faf5a-8a74-4080-b8f1-206c01aafd03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.226904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138faf5a-8a74-4080-b8f1-206c01aafd03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:40.229323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138faf5a-8a74-4080-b8f1-206c01aafd03 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:40.272094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 921/1846] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:40 2022] POST /identity/v3/projects => generated 367 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:40.273018 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-25eeca4b-9eb3-4e69-aabd-55236791d833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0281dafd87540349b33008e50028954'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:40.273268 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138faf5a-8a74-4080-b8f1-206c01aafd03 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:40.280778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:40.286631 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-25eeca4b-9eb3-4e69-aabd-55236791d833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0281dafd87540349b33008e50028954'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:40.287369 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-25eeca4b-9eb3-4e69-aabd-55236791d833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0281dafd87540349b33008e50028954'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:40.299720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-25eeca4b-9eb3-4e69-aabd-55236791d833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0281dafd87540349b33008e50028954'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:40.322485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-25eeca4b-9eb3-4e69-aabd-55236791d833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0281dafd87540349b33008e50028954'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:40.343915 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 926/1847] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:40 2022] DELETE /identity/v3/users/c0281dafd87540349b33008e50028954 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:40.346121 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a360b51-c21a-406a-9ec5-173d37b195ae None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.346615 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a360b51-c21a-406a-9ec5-173d37b195ae None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.347684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a360b51-c21a-406a-9ec5-173d37b195ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.410533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-6a360b51-c21a-406a-9ec5-173d37b195ae None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:40.411708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6a360b51-c21a-406a-9ec5-173d37b195ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:40.478475 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138faf5a-8a74-4080-b8f1-206c01aafd03 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:40.505405 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:40.537381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 927/1848] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:40.541602 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a360b51-c21a-406a-9ec5-173d37b195ae None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:40.543791 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6a360b51-c21a-406a-9ec5-173d37b195ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:40.596538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a360b51-c21a-406a-9ec5-173d37b195ae None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:40.619694 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cabcd-f142-4e69-8ad8-c6790a093a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:40.622903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cabcd-f142-4e69-8ad8-c6790a093a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.623233 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cabcd-f142-4e69-8ad8-c6790a093a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.629568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cabcd-f142-4e69-8ad8-c6790a093a97 admin admin] PATH_INFO: `/v3/projects/1e82de68ad9c430ba48b04b8e366c746` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.647924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cabcd-f142-4e69-8ad8-c6790a093a97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e82de68ad9c430ba48b04b8e366c746)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:40.651168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cabcd-f142-4e69-8ad8-c6790a093a97 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:40.681763 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873d2a4-30b9-429e-a0c0-f3afcde4e36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:40.684619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873d2a4-30b9-429e-a0c0-f3afcde4e36f admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.686253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873d2a4-30b9-429e-a0c0-f3afcde4e36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.686686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873d2a4-30b9-429e-a0c0-f3afcde4e36f admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.687867 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873d2a4-30b9-429e-a0c0-f3afcde4e36f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:40.690090 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873d2a4-30b9-429e-a0c0-f3afcde4e36f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:40.733212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 922/1849] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:40 2022] POST /identity/v3/users => generated 284 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:40.734626 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873d2a4-30b9-429e-a0c0-f3afcde4e36f admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:40.737021 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e873d2a4-30b9-429e-a0c0-f3afcde4e36f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:40.749141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1c0cabcd-f142-4e69-8ad8-c6790a093a97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e82de68ad9c430ba48b04b8e366c746', 'request_id': 'req-1c0cabcd-f142-4e69-8ad8-c6790a093a97'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:40.761987 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1c0cabcd-f142-4e69-8ad8-c6790a093a97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e82de68ad9c430ba48b04b8e366c746', 'request_id': 'req-1c0cabcd-f142-4e69-8ad8-c6790a093a97'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:40.776913 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1c0cabcd-f142-4e69-8ad8-c6790a093a97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e82de68ad9c430ba48b04b8e366c746', 'request_id': 'req-1c0cabcd-f142-4e69-8ad8-c6790a093a97'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:40.809140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1c0cabcd-f142-4e69-8ad8-c6790a093a97 admin admin] The token cache is being invalidate because project 1e82de68ad9c430ba48b04b8e366c746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:40.809942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1c0cabcd-f142-4e69-8ad8-c6790a093a97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:40.812692 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1c0cabcd-f142-4e69-8ad8-c6790a093a97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:40.814432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 928/1850] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:40 2022] DELETE /identity/v3/projects/1e82de68ad9c430ba48b04b8e366c746 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:40.815320 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cabcd-f142-4e69-8ad8-c6790a093a97 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:40.819162 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1c0cabcd-f142-4e69-8ad8-c6790a093a97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:40.852270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873d2a4-30b9-429e-a0c0-f3afcde4e36f admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:40.985605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247ff73-bdd9-4a94-9e81-5c5dc77e84a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:40.987000 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247ff73-bdd9-4a94-9e81-5c5dc77e84a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.987393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247ff73-bdd9-4a94-9e81-5c5dc77e84a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.987778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247ff73-bdd9-4a94-9e81-5c5dc77e84a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.991749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f247ff73-bdd9-4a94-9e81-5c5dc77e84a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:41.012521 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247ff73-bdd9-4a94-9e81-5c5dc77e84a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.012932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cabcd-f142-4e69-8ad8-c6790a093a97 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.014717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247ff73-bdd9-4a94-9e81-5c5dc77e84a2 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.022665 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:41.026413 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6ffbb-8928-4c74-8738-f9f59bcff667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:41.027139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6ffbb-8928-4c74-8738-f9f59bcff667 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.027431 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6ffbb-8928-4c74-8738-f9f59bcff667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.027715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6ffbb-8928-4c74-8738-f9f59bcff667 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.028524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6ffbb-8928-4c74-8738-f9f59bcff667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.029167 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 923/1851] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:41.030467 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247ff73-bdd9-4a94-9e81-5c5dc77e84a2 service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:41.030591 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6ffbb-8928-4c74-8738-f9f59bcff667 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.032622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f247ff73-bdd9-4a94-9e81-5c5dc77e84a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:41.055263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 929/1852] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:41.055497 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247ff73-bdd9-4a94-9e81-5c5dc77e84a2 service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.056452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6ffbb-8928-4c74-8738-f9f59bcff667 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:41.064618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961cccb3-cce5-4934-949e-ffc3e8c09f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:41.065385 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cccb3-cce5-4934-949e-ffc3e8c09f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.065942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cccb3-cce5-4934-949e-ffc3e8c09f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.066576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cccb3-cce5-4934-949e-ffc3e8c09f98 admin admin] PATH_INFO: `/v3/users/4bc38af54a4c4ec3a9041d1d624b987a` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.070911 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961cccb3-cce5-4934-949e-ffc3e8c09f98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bc38af54a4c4ec3a9041d1d624b987a)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.074984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961cccb3-cce5-4934-949e-ffc3e8c09f98 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.081046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6ffbb-8928-4c74-8738-f9f59bcff667 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.093835 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0697066-37a7-4c93-9e50-2e80708ef734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:41.094795 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0697066-37a7-4c93-9e50-2e80708ef734 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.094795 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0697066-37a7-4c93-9e50-2e80708ef734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.095070 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0697066-37a7-4c93-9e50-2e80708ef734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.096089 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0697066-37a7-4c93-9e50-2e80708ef734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.098088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0697066-37a7-4c93-9e50-2e80708ef734 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.121809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 930/1853] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:41.126614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0697066-37a7-4c93-9e50-2e80708ef734 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:41.145688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0697066-37a7-4c93-9e50-2e80708ef734 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.162784 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d07e48-8ae3-4433-a4f3-6e7178fb3b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:41.163705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d07e48-8ae3-4433-a4f3-6e7178fb3b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.164213 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d07e48-8ae3-4433-a4f3-6e7178fb3b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.164843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d07e48-8ae3-4433-a4f3-6e7178fb3b33 admin admin] PATH_INFO: `/v3/projects/9700a04efbbc4c3fbaac0fc5ed2e091f/users/ac3ee12752604c0eb49846f07ec920d9/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.172231 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-961cccb3-cce5-4934-949e-ffc3e8c09f98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bc38af54a4c4ec3a9041d1d624b987a'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.185396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-961cccb3-cce5-4934-949e-ffc3e8c09f98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bc38af54a4c4ec3a9041d1d624b987a'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.199250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d07e48-8ae3-4433-a4f3-6e7178fb3b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9700a04efbbc4c3fbaac0fc5ed2e091f, user_id=ac3ee12752604c0eb49846f07ec920d9, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.202280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d07e48-8ae3-4433-a4f3-6e7178fb3b33 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.215868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-961cccb3-cce5-4934-949e-ffc3e8c09f98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bc38af54a4c4ec3a9041d1d624b987a'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.221829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 931/1854] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:41 2022] PUT /identity/v3/projects/9700a04efbbc4c3fbaac0fc5ed2e091f/users/ac3ee12752604c0eb49846f07ec920d9/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:41.224328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-961cccb3-cce5-4934-949e-ffc3e8c09f98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bc38af54a4c4ec3a9041d1d624b987a'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.225309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-961cccb3-cce5-4934-949e-ffc3e8c09f98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bc38af54a4c4ec3a9041d1d624b987a'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.236792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62898cdb-c19d-4625-963e-3bfdd6a5d37a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.237086 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62898cdb-c19d-4625-963e-3bfdd6a5d37a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.237370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62898cdb-c19d-4625-963e-3bfdd6a5d37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.239102 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 924/1855] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:41 2022] DELETE /identity/v3/users/4bc38af54a4c4ec3a9041d1d624b987a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:41.283921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-62898cdb-c19d-4625-963e-3bfdd6a5d37a None None] MFA Rules not processed for user `ac3ee12752604c0eb49846f07ec920d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:41.285212 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-62898cdb-c19d-4625-963e-3bfdd6a5d37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:41.303508 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:41.365618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 932/1856] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:41 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:41.366778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961cccb3-cce5-4934-949e-ffc3e8c09f98 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:41.368576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-961cccb3-cce5-4934-949e-ffc3e8c09f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:41.381559 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969627a-b79b-4320-9f76-a2c22b3353bd None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.384635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969627a-b79b-4320-9f76-a2c22b3353bd None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.385625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969627a-b79b-4320-9f76-a2c22b3353bd None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.389628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 933/1857] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jul 20 19:24:41 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:24:41.426656 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025ad5e-b20a-4b4e-a49d-c5dba8051bbc None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.427871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025ad5e-b20a-4b4e-a49d-c5dba8051bbc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.428732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025ad5e-b20a-4b4e-a49d-c5dba8051bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.466452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-9025ad5e-b20a-4b4e-a49d-c5dba8051bbc None None] MFA Rules not processed for user `267b178236e046e4bc694b2da6f86eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:41.469406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9025ad5e-b20a-4b4e-a49d-c5dba8051bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:41.523528 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961cccb3-cce5-4934-949e-ffc3e8c09f98 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.531046 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c878f9e-908d-43c3-b41e-7f74fdc24a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:41.531989 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c878f9e-908d-43c3-b41e-7f74fdc24a1c admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.532537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c878f9e-908d-43c3-b41e-7f74fdc24a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.533125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c878f9e-908d-43c3-b41e-7f74fdc24a1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.534713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c878f9e-908d-43c3-b41e-7f74fdc24a1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.537625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c878f9e-908d-43c3-b41e-7f74fdc24a1c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.598903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 925/1858] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:41 2022] POST /identity/v3/projects => generated 365 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:41.600613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c878f9e-908d-43c3-b41e-7f74fdc24a1c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:41.631094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 934/1859] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jul 20 19:24:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:41.648896 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025ad5e-b20a-4b4e-a49d-c5dba8051bbc None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:41.654427 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9025ad5e-b20a-4b4e-a49d-c5dba8051bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:41.696534 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025ad5e-b20a-4b4e-a49d-c5dba8051bbc None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.699629 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:41.710159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c878f9e-908d-43c3-b41e-7f74fdc24a1c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.742992 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f21a6-490c-4aba-a9e8-45b68c9b1233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:41.743694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f21a6-490c-4aba-a9e8-45b68c9b1233 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.743981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f21a6-490c-4aba-a9e8-45b68c9b1233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.744256 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f21a6-490c-4aba-a9e8-45b68c9b1233 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.744994 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f21a6-490c-4aba-a9e8-45b68c9b1233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.753386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f21a6-490c-4aba-a9e8-45b68c9b1233 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.791707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 926/1860] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:41 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:41.793123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f21a6-490c-4aba-a9e8-45b68c9b1233 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:41.795351 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-b64f21a6-490c-4aba-a9e8-45b68c9b1233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:41.809590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f21a6-490c-4aba-a9e8-45b68c9b1233 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.819189 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e360f-5332-4c8f-9404-7291820b2667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:41.820247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e360f-5332-4c8f-9404-7291820b2667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.820917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e360f-5332-4c8f-9404-7291820b2667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.821044 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e360f-5332-4c8f-9404-7291820b2667 admin admin] PATH_INFO: `/v3/projects/e4f3accc405047b19af7e3a5a4d23261` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.822377 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e360f-5332-4c8f-9404-7291820b2667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4f3accc405047b19af7e3a5a4d23261)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.825268 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e360f-5332-4c8f-9404-7291820b2667 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.842338 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce32196-2334-47f7-a9af-dcaf8f8cf453 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:41.843404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce32196-2334-47f7-a9af-dcaf8f8cf453 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.844348 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce32196-2334-47f7-a9af-dcaf8f8cf453 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.845200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce32196-2334-47f7-a9af-dcaf8f8cf453 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.857363 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce32196-2334-47f7-a9af-dcaf8f8cf453 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.859352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce32196-2334-47f7-a9af-dcaf8f8cf453 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.874065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 935/1861] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:41.886566 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce32196-2334-47f7-a9af-dcaf8f8cf453 service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:41.923530 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-962e360f-5332-4c8f-9404-7291820b2667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4f3accc405047b19af7e3a5a4d23261', 'request_id': 'req-962e360f-5332-4c8f-9404-7291820b2667'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.934490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce32196-2334-47f7-a9af-dcaf8f8cf453 service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.952717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-962e360f-5332-4c8f-9404-7291820b2667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4f3accc405047b19af7e3a5a4d23261', 'request_id': 'req-962e360f-5332-4c8f-9404-7291820b2667'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.978278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-962e360f-5332-4c8f-9404-7291820b2667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4f3accc405047b19af7e3a5a4d23261', 'request_id': 'req-962e360f-5332-4c8f-9404-7291820b2667'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.994108 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:42.007791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-962e360f-5332-4c8f-9404-7291820b2667 admin admin] The token cache is being invalidate because project e4f3accc405047b19af7e3a5a4d23261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:42.008113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-962e360f-5332-4c8f-9404-7291820b2667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.010779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-962e360f-5332-4c8f-9404-7291820b2667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.016824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 927/1862] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:41 2022] DELETE /identity/v3/projects/e4f3accc405047b19af7e3a5a4d23261 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:42.016824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e360f-5332-4c8f-9404-7291820b2667 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:42.018348 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-962e360f-5332-4c8f-9404-7291820b2667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:42.094060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87850da-0b87-4f58-a086-bd97ed3d00f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:42.094776 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87850da-0b87-4f58-a086-bd97ed3d00f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.095060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87850da-0b87-4f58-a086-bd97ed3d00f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.095337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87850da-0b87-4f58-a086-bd97ed3d00f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.096078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87850da-0b87-4f58-a086-bd97ed3d00f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:42.099711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87850da-0b87-4f58-a086-bd97ed3d00f6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:42.115843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 936/1863] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:42.127261 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98b6bc-262b-40d3-8691-ea4ade7ddd32 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.127585 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98b6bc-262b-40d3-8691-ea4ade7ddd32 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.127878 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98b6bc-262b-40d3-8691-ea4ade7ddd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.144817 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e360f-5332-4c8f-9404-7291820b2667 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:42.161836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc0031-9b31-47bd-9202-66125d238736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:42.162736 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc0031-9b31-47bd-9202-66125d238736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.163057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc0031-9b31-47bd-9202-66125d238736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.163295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc0031-9b31-47bd-9202-66125d238736 admin admin] PATH_INFO: `/v3/projects/1e1732faff4e4e34a9483316eccdb17b` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.170197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc0031-9b31-47bd-9202-66125d238736 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e1732faff4e4e34a9483316eccdb17b)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:42.172195 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc0031-9b31-47bd-9202-66125d238736 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:42.177451 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ec98b6bc-262b-40d3-8691-ea4ade7ddd32 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:42.179039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ec98b6bc-262b-40d3-8691-ea4ade7ddd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:42.201244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 937/1864] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:42.203409 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98b6bc-262b-40d3-8691-ea4ade7ddd32 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:42.205845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ec98b6bc-262b-40d3-8691-ea4ade7ddd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:42.226593 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98b6bc-262b-40d3-8691-ea4ade7ddd32 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:42.232384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd649c-23a4-474f-9c91-2d7573bd58d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:42.233356 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd649c-23a4-474f-9c91-2d7573bd58d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.233636 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd649c-23a4-474f-9c91-2d7573bd58d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.233917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd649c-23a4-474f-9c91-2d7573bd58d2 admin admin] PATH_INFO: `/v3/projects/e5ef5f43829f4a499d5b006b47fa22f8/users/d137e95588d14319964e1e8578c2954a/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.257062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd649c-23a4-474f-9c91-2d7573bd58d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ef5f43829f4a499d5b006b47fa22f8, user_id=d137e95588d14319964e1e8578c2954a, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:42.260828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd649c-23a4-474f-9c91-2d7573bd58d2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:42.293338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-21dc0031-9b31-47bd-9202-66125d238736 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e1732faff4e4e34a9483316eccdb17b', 'request_id': 'req-21dc0031-9b31-47bd-9202-66125d238736'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.295714 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 938/1865] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:42 2022] PUT /identity/v3/projects/e5ef5f43829f4a499d5b006b47fa22f8/users/d137e95588d14319964e1e8578c2954a/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:42.297344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137b577-c7bd-40a5-a0ba-de46e7964ba8 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.298924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137b577-c7bd-40a5-a0ba-de46e7964ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.299224 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137b577-c7bd-40a5-a0ba-de46e7964ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.317109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-21dc0031-9b31-47bd-9202-66125d238736 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e1732faff4e4e34a9483316eccdb17b', 'request_id': 'req-21dc0031-9b31-47bd-9202-66125d238736'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.327102 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-21dc0031-9b31-47bd-9202-66125d238736 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e1732faff4e4e34a9483316eccdb17b', 'request_id': 'req-21dc0031-9b31-47bd-9202-66125d238736'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.345874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=81,keystone:INSERT=5,keystone:DELETE=14 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:42.347048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c137b577-c7bd-40a5-a0ba-de46e7964ba8 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:42.349330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c137b577-c7bd-40a5-a0ba-de46e7964ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:42.356358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-21dc0031-9b31-47bd-9202-66125d238736 admin admin] The token cache is being invalidate because project 1e1732faff4e4e34a9483316eccdb17b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:42.357409 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-21dc0031-9b31-47bd-9202-66125d238736 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.371146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-21dc0031-9b31-47bd-9202-66125d238736 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.372991 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 928/1866] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:42 2022] DELETE /identity/v3/projects/1e1732faff4e4e34a9483316eccdb17b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:42.373984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc0031-9b31-47bd-9202-66125d238736 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:42.376513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-21dc0031-9b31-47bd-9202-66125d238736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:42.436509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 939/1867] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:42.436509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137b577-c7bd-40a5-a0ba-de46e7964ba8 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:42.436509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c137b577-c7bd-40a5-a0ba-de46e7964ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:42.458401 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc0031-9b31-47bd-9202-66125d238736 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:42.465687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd8cde-25c3-4ced-9c0f-04ac30aaed3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:42.466690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd8cde-25c3-4ced-9c0f-04ac30aaed3d admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.467157 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd8cde-25c3-4ced-9c0f-04ac30aaed3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.467544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd8cde-25c3-4ced-9c0f-04ac30aaed3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.468480 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd8cde-25c3-4ced-9c0f-04ac30aaed3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:42.470884 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd8cde-25c3-4ced-9c0f-04ac30aaed3d admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:42.488905 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 929/1868] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:42.489715 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd8cde-25c3-4ced-9c0f-04ac30aaed3d admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:42.492141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137b577-c7bd-40a5-a0ba-de46e7964ba8 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:42.500926 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd8cde-25c3-4ced-9c0f-04ac30aaed3d admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:42.503197 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658c513-c9cf-4ee5-8fe5-6460bf878315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:42.504168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658c513-c9cf-4ee5-8fe5-6460bf878315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.504483 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658c513-c9cf-4ee5-8fe5-6460bf878315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.504764 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658c513-c9cf-4ee5-8fe5-6460bf878315 admin admin] PATH_INFO: `/v3/users/16bf7a4c80e24970a80e1d8dc45ec301` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.506782 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658c513-c9cf-4ee5-8fe5-6460bf878315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16bf7a4c80e24970a80e1d8dc45ec301)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:42.507928 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124064ac-d95d-44f2-a53d-fa37e8270936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:42.508439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124064ac-d95d-44f2-a53d-fa37e8270936 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.508730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124064ac-d95d-44f2-a53d-fa37e8270936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.508846 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658c513-c9cf-4ee5-8fe5-6460bf878315 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:42.509595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124064ac-d95d-44f2-a53d-fa37e8270936 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.509777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124064ac-d95d-44f2-a53d-fa37e8270936 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:42.510436 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124064ac-d95d-44f2-a53d-fa37e8270936 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:42.512062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124064ac-d95d-44f2-a53d-fa37e8270936 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:42.529088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 930/1869] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:42.529088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124064ac-d95d-44f2-a53d-fa37e8270936 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:42.537822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124064ac-d95d-44f2-a53d-fa37e8270936 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:42.543381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8b09e-5af7-4edc-86f0-e5f2c528c6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:42.544547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8b09e-5af7-4edc-86f0-e5f2c528c6ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.544857 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8b09e-5af7-4edc-86f0-e5f2c528c6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.545126 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8b09e-5af7-4edc-86f0-e5f2c528c6ec admin admin] PATH_INFO: `/v3/projects/e5ef5f43829f4a499d5b006b47fa22f8/users/d137e95588d14319964e1e8578c2954a/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.547931 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8b09e-5af7-4edc-86f0-e5f2c528c6ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ef5f43829f4a499d5b006b47fa22f8, user_id=d137e95588d14319964e1e8578c2954a, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:42.550787 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8b09e-5af7-4edc-86f0-e5f2c528c6ec admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:42.565279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 931/1870] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:42 2022] PUT /identity/v3/projects/e5ef5f43829f4a499d5b006b47fa22f8/users/d137e95588d14319964e1e8578c2954a/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:42.566511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8b09e-5af7-4edc-86f0-e5f2c528c6ec admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:42.567593 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1658c513-c9cf-4ee5-8fe5-6460bf878315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16bf7a4c80e24970a80e1d8dc45ec301'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.577961 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1658c513-c9cf-4ee5-8fe5-6460bf878315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16bf7a4c80e24970a80e1d8dc45ec301'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.578736 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1658c513-c9cf-4ee5-8fe5-6460bf878315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16bf7a4c80e24970a80e1d8dc45ec301'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.588760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1658c513-c9cf-4ee5-8fe5-6460bf878315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16bf7a4c80e24970a80e1d8dc45ec301'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.605070 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1658c513-c9cf-4ee5-8fe5-6460bf878315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16bf7a4c80e24970a80e1d8dc45ec301'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.612247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 940/1871] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:42 2022] DELETE /identity/v3/users/16bf7a4c80e24970a80e1d8dc45ec301 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:42.613616 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa8bcc-9bea-4925-b1a3-d825985c09f2 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.613975 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa8bcc-9bea-4925-b1a3-d825985c09f2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.614187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa8bcc-9bea-4925-b1a3-d825985c09f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.640265 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8b09e-5af7-4edc-86f0-e5f2c528c6ec admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:42.656380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:42.663505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2dfa8bcc-9bea-4925-b1a3-d825985c09f2 None None] MFA Rules not processed for user `d137e95588d14319964e1e8578c2954a`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:42.665064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2dfa8bcc-9bea-4925-b1a3-d825985c09f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:42.718173 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce2037-9218-4e8f-a118-e0b278aded48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:42.718877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce2037-9218-4e8f-a118-e0b278aded48 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.719522 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce2037-9218-4e8f-a118-e0b278aded48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.721249 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce2037-9218-4e8f-a118-e0b278aded48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.723208 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce2037-9218-4e8f-a118-e0b278aded48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:42.726922 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce2037-9218-4e8f-a118-e0b278aded48 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:42.785158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 932/1872] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:42 2022] POST /identity/v3/projects => generated 379 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:42.789535 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce2037-9218-4e8f-a118-e0b278aded48 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:42.795117 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-86ce2037-9218-4e8f-a118-e0b278aded48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:42.803301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:42.843741 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 941/1873] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:42 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:42.855607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa8bcc-9bea-4925-b1a3-d825985c09f2 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:42.857122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2dfa8bcc-9bea-4925-b1a3-d825985c09f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:42.906154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce2037-9218-4e8f-a118-e0b278aded48 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:42.910960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93975fb0-1607-43fe-9230-c079707afd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:42.911672 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93975fb0-1607-43fe-9230-c079707afd8e admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.911993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93975fb0-1607-43fe-9230-c079707afd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.912271 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93975fb0-1607-43fe-9230-c079707afd8e admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.913004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93975fb0-1607-43fe-9230-c079707afd8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:42.915133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93975fb0-1607-43fe-9230-c079707afd8e admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:42.920953 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa8bcc-9bea-4925-b1a3-d825985c09f2 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:42.941127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 933/1874] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:42 2022] POST /identity/v3/users => generated 290 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:42.941998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93975fb0-1607-43fe-9230-c079707afd8e admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:42.957520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93975fb0-1607-43fe-9230-c079707afd8e admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:42.965735 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738547eb-0b33-4467-89a1-fa503f5f12c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:42.971451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738547eb-0b33-4467-89a1-fa503f5f12c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.972654 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738547eb-0b33-4467-89a1-fa503f5f12c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.973485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738547eb-0b33-4467-89a1-fa503f5f12c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.975658 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738547eb-0b33-4467-89a1-fa503f5f12c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:42.978581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738547eb-0b33-4467-89a1-fa503f5f12c0 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:42.998232 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 934/1875] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:43.001034 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738547eb-0b33-4467-89a1-fa503f5f12c0 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.004524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c420d-e064-4ef5-93ad-68dd002ee262 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.005251 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-738547eb-0b33-4467-89a1-fa503f5f12c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:43.006392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c420d-e064-4ef5-93ad-68dd002ee262 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.006785 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c420d-e064-4ef5-93ad-68dd002ee262 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.007167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c420d-e064-4ef5-93ad-68dd002ee262 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.024664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738547eb-0b33-4467-89a1-fa503f5f12c0 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.034891 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64d7f2-9146-4670-872d-afd7b7ff502f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.035131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64d7f2-9146-4670-872d-afd7b7ff502f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.035428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64d7f2-9146-4670-872d-afd7b7ff502f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.035707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64d7f2-9146-4670-872d-afd7b7ff502f admin admin] PATH_INFO: `/v3/projects/e4d529e97c914403a74cf39ecda22201` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.037099 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64d7f2-9146-4670-872d-afd7b7ff502f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4d529e97c914403a74cf39ecda22201)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.038918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64d7f2-9146-4670-872d-afd7b7ff502f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.102390 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c420d-e064-4ef5-93ad-68dd002ee262 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.102602 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6d64d7f2-9146-4670-872d-afd7b7ff502f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4d529e97c914403a74cf39ecda22201', 'request_id': 'req-6d64d7f2-9146-4670-872d-afd7b7ff502f'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:43.103886 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c420d-e064-4ef5-93ad-68dd002ee262 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.118954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 942/1876] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:42 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:43.119682 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c420d-e064-4ef5-93ad-68dd002ee262 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.120714 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6d64d7f2-9146-4670-872d-afd7b7ff502f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4d529e97c914403a74cf39ecda22201', 'request_id': 'req-6d64d7f2-9146-4670-872d-afd7b7ff502f'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:43.125935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:43.129512 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6d64d7f2-9146-4670-872d-afd7b7ff502f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4d529e97c914403a74cf39ecda22201', 'request_id': 'req-6d64d7f2-9146-4670-872d-afd7b7ff502f'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:43.133155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c420d-e064-4ef5-93ad-68dd002ee262 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.151326 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6d64d7f2-9146-4670-872d-afd7b7ff502f admin admin] The token cache is being invalidate because project e4d529e97c914403a74cf39ecda22201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:43.152663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6d64d7f2-9146-4670-872d-afd7b7ff502f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:43.155316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6d64d7f2-9146-4670-872d-afd7b7ff502f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:43.159651 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 935/1877] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:42 2022] DELETE /identity/v3/projects/e4d529e97c914403a74cf39ecda22201 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:43.164739 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72feec93-6521-4260-97cf-bd0cc40e7696 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.165025 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72feec93-6521-4260-97cf-bd0cc40e7696 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.165337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72feec93-6521-4260-97cf-bd0cc40e7696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.196170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-72feec93-6521-4260-97cf-bd0cc40e7696 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:43.197845 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-72feec93-6521-4260-97cf-bd0cc40e7696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:43.256552 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814c81b-a6c1-4590-a680-cd6c9ff62100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.257920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814c81b-a6c1-4590-a680-cd6c9ff62100 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.258367 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814c81b-a6c1-4590-a680-cd6c9ff62100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.259095 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814c81b-a6c1-4590-a680-cd6c9ff62100 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.260429 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b814c81b-a6c1-4590-a680-cd6c9ff62100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.262767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b814c81b-a6c1-4590-a680-cd6c9ff62100 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.277883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 936/1878] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:43.278923 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72feec93-6521-4260-97cf-bd0cc40e7696 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.280014 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 943/1879] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:43.283745 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-72feec93-6521-4260-97cf-bd0cc40e7696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:43.284950 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814c81b-a6c1-4590-a680-cd6c9ff62100 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.286851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b814c81b-a6c1-4590-a680-cd6c9ff62100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:43.292013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:43.305575 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814c81b-a6c1-4590-a680-cd6c9ff62100 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.311440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17dbe09-52c8-4654-b7d8-652f7b2c78f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.313266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17dbe09-52c8-4654-b7d8-652f7b2c78f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.313703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17dbe09-52c8-4654-b7d8-652f7b2c78f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.314668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17dbe09-52c8-4654-b7d8-652f7b2c78f3 admin admin] PATH_INFO: `/v3/projects/852ef6f37fd14eb4845270afb2328e32/users/ca6dac26fbce4f6aae29633a0bcf4d87/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.332481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17dbe09-52c8-4654-b7d8-652f7b2c78f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=852ef6f37fd14eb4845270afb2328e32, user_id=ca6dac26fbce4f6aae29633a0bcf4d87, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.335356 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17dbe09-52c8-4654-b7d8-652f7b2c78f3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.365645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 944/1880] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:43 2022] PUT /identity/v3/projects/852ef6f37fd14eb4845270afb2328e32/users/ca6dac26fbce4f6aae29633a0bcf4d87/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:43.367174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023fbbd-85bd-4760-b7c3-785a55e503fa None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.367474 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023fbbd-85bd-4760-b7c3-785a55e503fa None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.367746 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023fbbd-85bd-4760-b7c3-785a55e503fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.388474 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72feec93-6521-4260-97cf-bd0cc40e7696 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.388695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e023fbbd-85bd-4760-b7c3-785a55e503fa None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:43.389897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e023fbbd-85bd-4760-b7c3-785a55e503fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:43.454469 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fcc84-83bc-4eae-a92e-e763d3d076b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.455792 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fcc84-83bc-4eae-a92e-e763d3d076b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.456326 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fcc84-83bc-4eae-a92e-e763d3d076b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.456881 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fcc84-83bc-4eae-a92e-e763d3d076b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.460935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-9d9fcc84-83bc-4eae-a92e-e763d3d076b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:43.477552 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 945/1881] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:43.481653 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0956d5e-bc06-4310-8362-28416fa46dbd None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.481953 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0956d5e-bc06-4310-8362-28416fa46dbd None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.482233 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0956d5e-bc06-4310-8362-28416fa46dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.517990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c0956d5e-bc06-4310-8362-28416fa46dbd None None] MFA Rules not processed for user `ca6dac26fbce4f6aae29633a0bcf4d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:43.519606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c0956d5e-bc06-4310-8362-28416fa46dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:43.551014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fcc84-83bc-4eae-a92e-e763d3d076b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.553091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fcc84-83bc-4eae-a92e-e763d3d076b4 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.565172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 937/1882] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:43 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:43.565985 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fcc84-83bc-4eae-a92e-e763d3d076b4 service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.589420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fcc84-83bc-4eae-a92e-e763d3d076b4 service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.596193 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad324052-dcd3-42ec-b61b-7386d80cf4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.597814 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad324052-dcd3-42ec-b61b-7386d80cf4dd admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.598111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad324052-dcd3-42ec-b61b-7386d80cf4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.598395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad324052-dcd3-42ec-b61b-7386d80cf4dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.600208 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad324052-dcd3-42ec-b61b-7386d80cf4dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:43.600564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad324052-dcd3-42ec-b61b-7386d80cf4dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.609460 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad324052-dcd3-42ec-b61b-7386d80cf4dd admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.622935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 938/1883] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:43.627174 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad324052-dcd3-42ec-b61b-7386d80cf4dd admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.649418 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad324052-dcd3-42ec-b61b-7386d80cf4dd admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.656686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e616028-2bcd-40fd-a821-6b65122be3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.657415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e616028-2bcd-40fd-a821-6b65122be3ec admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.657700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e616028-2bcd-40fd-a821-6b65122be3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.657978 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e616028-2bcd-40fd-a821-6b65122be3ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.658966 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e616028-2bcd-40fd-a821-6b65122be3ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.661157 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e616028-2bcd-40fd-a821-6b65122be3ec admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.674187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 946/1884] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:43.680168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0956d5e-bc06-4310-8362-28416fa46dbd None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.699955 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0956d5e-bc06-4310-8362-28416fa46dbd None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.706122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d83b5-39ad-4a40-9052-0093013bed15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.707197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 939/1885] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:43 2022] POST /identity/v3/projects => generated 393 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:43.707742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d83b5-39ad-4a40-9052-0093013bed15 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.708095 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e616028-2bcd-40fd-a821-6b65122be3ec admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.708210 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d83b5-39ad-4a40-9052-0093013bed15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.708546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d83b5-39ad-4a40-9052-0093013bed15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.710405 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d83b5-39ad-4a40-9052-0093013bed15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.712717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d83b5-39ad-4a40-9052-0093013bed15 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.719572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:43.756209 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 947/1886] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:43 2022] POST /identity/v3/projects => generated 381 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:43.757322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d83b5-39ad-4a40-9052-0093013bed15 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.774959 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:43.815005 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e616028-2bcd-40fd-a821-6b65122be3ec admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.890126 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4df1f-8ef0-49ac-a9eb-598961096880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.891078 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4df1f-8ef0-49ac-a9eb-598961096880 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.891462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4df1f-8ef0-49ac-a9eb-598961096880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.891841 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4df1f-8ef0-49ac-a9eb-598961096880 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.892824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4df1f-8ef0-49ac-a9eb-598961096880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.896772 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4df1f-8ef0-49ac-a9eb-598961096880 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.898294 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d83b5-39ad-4a40-9052-0093013bed15 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.903516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61d862-8577-41a1-bba1-7d32d45ea2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.904638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61d862-8577-41a1-bba1-7d32d45ea2a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.905413 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61d862-8577-41a1-bba1-7d32d45ea2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.905413 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61d862-8577-41a1-bba1-7d32d45ea2a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.906605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61d862-8577-41a1-bba1-7d32d45ea2a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.908493 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61d862-8577-41a1-bba1-7d32d45ea2a2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.922152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 940/1887] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:43 2022] POST /identity/v3/users => generated 297 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:43.926812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4df1f-8ef0-49ac-a9eb-598961096880 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.939881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 948/1888] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:43 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:43.940806 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4df1f-8ef0-49ac-a9eb-598961096880 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.941421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61d862-8577-41a1-bba1-7d32d45ea2a2 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.946941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0bd75-3b83-4974-a30c-bcd7c13cefca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.947765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0bd75-3b83-4974-a30c-bcd7c13cefca admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.948027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0bd75-3b83-4974-a30c-bcd7c13cefca admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.948410 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0bd75-3b83-4974-a30c-bcd7c13cefca admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.949258 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0bd75-3b83-4974-a30c-bcd7c13cefca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.950919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0bd75-3b83-4974-a30c-bcd7c13cefca admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.954567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61d862-8577-41a1-bba1-7d32d45ea2a2 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.964407 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564dc10-e357-4ae6-90a4-f923231436c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.964407 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564dc10-e357-4ae6-90a4-f923231436c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.964407 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564dc10-e357-4ae6-90a4-f923231436c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.964407 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564dc10-e357-4ae6-90a4-f923231436c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.965571 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 941/1889] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:43.965651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564dc10-e357-4ae6-90a4-f923231436c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.965651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564dc10-e357-4ae6-90a4-f923231436c4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.967059 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0bd75-3b83-4974-a30c-bcd7c13cefca admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.979570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0bd75-3b83-4974-a30c-bcd7c13cefca admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.982445 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 949/1890] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:43.985408 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecff75-710f-428e-aec4-874cf6221ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.986195 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecff75-710f-428e-aec4-874cf6221ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.986581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecff75-710f-428e-aec4-874cf6221ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.986957 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecff75-710f-428e-aec4-874cf6221ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.987419 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564dc10-e357-4ae6-90a4-f923231436c4 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.988373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecff75-710f-428e-aec4-874cf6221ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.990630 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecff75-710f-428e-aec4-874cf6221ee3 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.001910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564dc10-e357-4ae6-90a4-f923231436c4 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.006907 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 942/1891] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:44.010435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadc7e5-eeb2-4960-ae08-ebb67037a233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.010812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecff75-710f-428e-aec4-874cf6221ee3 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:44.011944 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadc7e5-eeb2-4960-ae08-ebb67037a233 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.012481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadc7e5-eeb2-4960-ae08-ebb67037a233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.013171 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadc7e5-eeb2-4960-ae08-ebb67037a233 admin admin] PATH_INFO: `/v3/projects/3d34fcca57cd4fa49363f76a36c00ffe/users/0be06ca30b694345930bda307adce08d/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.021791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecff75-710f-428e-aec4-874cf6221ee3 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.027397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fd510-aa04-43e9-9c06-f6a53f1db3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.028315 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fd510-aa04-43e9-9c06-f6a53f1db3d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.028734 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fd510-aa04-43e9-9c06-f6a53f1db3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.029024 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fd510-aa04-43e9-9c06-f6a53f1db3d9 admin admin] PATH_INFO: `/v3/projects/0b5883eef00c4f2d925f7d026c13e745/users/573f2b6df5c24856beedc84afcd92fbf/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.038300 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cadc7e5-eeb2-4960-ae08-ebb67037a233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d34fcca57cd4fa49363f76a36c00ffe, user_id=0be06ca30b694345930bda307adce08d, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.041484 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cadc7e5-eeb2-4960-ae08-ebb67037a233 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.045013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fd510-aa04-43e9-9c06-f6a53f1db3d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b5883eef00c4f2d925f7d026c13e745, user_id=573f2b6df5c24856beedc84afcd92fbf, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.048253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fd510-aa04-43e9-9c06-f6a53f1db3d9 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.057401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 950/1892] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:43 2022] PUT /identity/v3/projects/3d34fcca57cd4fa49363f76a36c00ffe/users/0be06ca30b694345930bda307adce08d/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:44.061042 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadc7e5-eeb2-4960-ae08-ebb67037a233 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:44.064865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 943/1893] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:44 2022] PUT /identity/v3/projects/0b5883eef00c4f2d925f7d026c13e745/users/573f2b6df5c24856beedc84afcd92fbf/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:44.076270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec90857-35c6-4d16-8b23-b3b058c3a9d5 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.076678 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec90857-35c6-4d16-8b23-b3b058c3a9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.077091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec90857-35c6-4d16-8b23-b3b058c3a9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.106044 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-5ec90857-35c6-4d16-8b23-b3b058c3a9d5 None None] MFA Rules not processed for user `573f2b6df5c24856beedc84afcd92fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:44.107287 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5ec90857-35c6-4d16-8b23-b3b058c3a9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:44.134458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadc7e5-eeb2-4960-ae08-ebb67037a233 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.176682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 944/1894] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:44.181162 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec90857-35c6-4d16-8b23-b3b058c3a9d5 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:44.182400 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:44.197848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84343b9-84e6-45d7-8a1b-6d2a21b3b58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.198517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84343b9-84e6-45d7-8a1b-6d2a21b3b58b admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.198801 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84343b9-84e6-45d7-8a1b-6d2a21b3b58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.199078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84343b9-84e6-45d7-8a1b-6d2a21b3b58b admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.199821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84343b9-84e6-45d7-8a1b-6d2a21b3b58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.201466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84343b9-84e6-45d7-8a1b-6d2a21b3b58b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.204891 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:44.212447 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 951/1895] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:44.216260 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84343b9-84e6-45d7-8a1b-6d2a21b3b58b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:44.229927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84343b9-84e6-45d7-8a1b-6d2a21b3b58b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.244772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e309c-017f-4f74-8ec1-707d8435046e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.246185 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e309c-017f-4f74-8ec1-707d8435046e admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.246579 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e309c-017f-4f74-8ec1-707d8435046e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.246930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e309c-017f-4f74-8ec1-707d8435046e admin admin] PATH_INFO: `/v3/projects/3d34fcca57cd4fa49363f76a36c00ffe/users/0be06ca30b694345930bda307adce08d/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.250111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e309c-017f-4f74-8ec1-707d8435046e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d34fcca57cd4fa49363f76a36c00ffe, user_id=0be06ca30b694345930bda307adce08d, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.253108 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e309c-017f-4f74-8ec1-707d8435046e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.266446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec90857-35c6-4d16-8b23-b3b058c3a9d5 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.266968 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 952/1896] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:44 2022] PUT /identity/v3/projects/3d34fcca57cd4fa49363f76a36c00ffe/users/0be06ca30b694345930bda307adce08d/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:44.278061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f8a95-657b-4c3b-8544-163861c0a130 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.278333 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f8a95-657b-4c3b-8544-163861c0a130 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.278542 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f8a95-657b-4c3b-8544-163861c0a130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.312687 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ab4f8a95-657b-4c3b-8544-163861c0a130 None None] MFA Rules not processed for user `0be06ca30b694345930bda307adce08d`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:44.313958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ab4f8a95-657b-4c3b-8544-163861c0a130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:44.341583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f65c3-3cae-4b93-aa8a-8c6cc985f8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.342263 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f65c3-3cae-4b93-aa8a-8c6cc985f8e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.342881 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f65c3-3cae-4b93-aa8a-8c6cc985f8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.342881 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f65c3-3cae-4b93-aa8a-8c6cc985f8e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.343529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f65c3-3cae-4b93-aa8a-8c6cc985f8e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.345660 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f65c3-3cae-4b93-aa8a-8c6cc985f8e0 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.382892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 945/1897] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:44 2022] POST /identity/v3/projects => generated 391 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:44.384264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f65c3-3cae-4b93-aa8a-8c6cc985f8e0 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:44.403265 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 953/1898] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:44.416941 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f8a95-657b-4c3b-8544-163861c0a130 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:44.482368 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f65c3-3cae-4b93-aa8a-8c6cc985f8e0 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.488136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfc5a7-1571-43b3-815e-5b506cbcc7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.489070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfc5a7-1571-43b3-815e-5b506cbcc7f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.489369 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfc5a7-1571-43b3-815e-5b506cbcc7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.489652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfc5a7-1571-43b3-815e-5b506cbcc7f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.490462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfc5a7-1571-43b3-815e-5b506cbcc7f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.492577 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfc5a7-1571-43b3-815e-5b506cbcc7f5 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.506606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f8a95-657b-4c3b-8544-163861c0a130 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.521901 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 946/1899] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:44 2022] POST /identity/v3/users => generated 302 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:44.525476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfc5a7-1571-43b3-815e-5b506cbcc7f5 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:44.541738 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfc5a7-1571-43b3-815e-5b506cbcc7f5 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.547492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4080d-a096-4efb-8156-ad1d92c25a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.548340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4080d-a096-4efb-8156-ad1d92c25a2c admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.548919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4080d-a096-4efb-8156-ad1d92c25a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.549210 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4080d-a096-4efb-8156-ad1d92c25a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.550063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4080d-a096-4efb-8156-ad1d92c25a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.551918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4080d-a096-4efb-8156-ad1d92c25a2c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.569083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 947/1900] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:44.570166 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4080d-a096-4efb-8156-ad1d92c25a2c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:44.578614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f08f2-676d-46df-8bde-42a53a7531b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.580463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f08f2-676d-46df-8bde-42a53a7531b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.580463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f08f2-676d-46df-8bde-42a53a7531b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.580715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f08f2-676d-46df-8bde-42a53a7531b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.599137 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4080d-a096-4efb-8156-ad1d92c25a2c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.610598 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:44.613257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35491914-fec8-4181-b282-2f93ae5db7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.614658 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35491914-fec8-4181-b282-2f93ae5db7b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.614658 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35491914-fec8-4181-b282-2f93ae5db7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.615268 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35491914-fec8-4181-b282-2f93ae5db7b9 admin admin] PATH_INFO: `/v3/projects/6f78dbc95e9c44d5a534da074822f6ab/users/f5a9c3808b2143fdb895ee8a633a7bc6/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.656360 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35491914-fec8-4181-b282-2f93ae5db7b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f78dbc95e9c44d5a534da074822f6ab, user_id=f5a9c3808b2143fdb895ee8a633a7bc6, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.661557 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35491914-fec8-4181-b282-2f93ae5db7b9 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.679703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f08f2-676d-46df-8bde-42a53a7531b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.681436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f08f2-676d-46df-8bde-42a53a7531b7 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.688808 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 948/1901] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:44 2022] PUT /identity/v3/projects/6f78dbc95e9c44d5a534da074822f6ab/users/f5a9c3808b2143fdb895ee8a633a7bc6/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:44.689730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35491914-fec8-4181-b282-2f93ae5db7b9 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:44.699373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 954/1902] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:44.705364 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:44.818590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35491914-fec8-4181-b282-2f93ae5db7b9 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.836999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe4c5c-0a8e-4f62-af3b-9112a025360c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.837701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe4c5c-0a8e-4f62-af3b-9112a025360c admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.837990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe4c5c-0a8e-4f62-af3b-9112a025360c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.838282 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe4c5c-0a8e-4f62-af3b-9112a025360c admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.840091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe4c5c-0a8e-4f62-af3b-9112a025360c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.844325 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe4c5c-0a8e-4f62-af3b-9112a025360c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.871544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 949/1903] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:44.876535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f08f2-676d-46df-8bde-42a53a7531b7 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:44.899642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f08f2-676d-46df-8bde-42a53a7531b7 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.906049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f0786-7af1-4b0e-a145-e95080eb5709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.907388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f0786-7af1-4b0e-a145-e95080eb5709 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.907946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f0786-7af1-4b0e-a145-e95080eb5709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.908358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f0786-7af1-4b0e-a145-e95080eb5709 admin admin] PATH_INFO: `/v3/projects/6f78dbc95e9c44d5a534da074822f6ab/users/f5a9c3808b2143fdb895ee8a633a7bc6/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.911234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f0786-7af1-4b0e-a145-e95080eb5709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f78dbc95e9c44d5a534da074822f6ab, user_id=f5a9c3808b2143fdb895ee8a633a7bc6, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.914177 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f0786-7af1-4b0e-a145-e95080eb5709 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.932892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 955/1904] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:44 2022] PUT /identity/v3/projects/6f78dbc95e9c44d5a534da074822f6ab/users/f5a9c3808b2143fdb895ee8a633a7bc6/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:44.939393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941225a3-1b70-4dea-b11a-4d507739c88e None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.939920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941225a3-1b70-4dea-b11a-4d507739c88e None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.940397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941225a3-1b70-4dea-b11a-4d507739c88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.994993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-941225a3-1b70-4dea-b11a-4d507739c88e None None] MFA Rules not processed for user `f5a9c3808b2143fdb895ee8a633a7bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:44.997485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-941225a3-1b70-4dea-b11a-4d507739c88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:45.167694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 950/1905] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:44 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:45.222181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f0786-7af1-4b0e-a145-e95080eb5709 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:45.225476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-264f0786-7af1-4b0e-a145-e95080eb5709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:45.295263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f0786-7af1-4b0e-a145-e95080eb5709 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:45.302263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415fb27-d316-41ca-8066-3afd7d46ed1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:45.303618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415fb27-d316-41ca-8066-3afd7d46ed1b service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:45.303927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415fb27-d316-41ca-8066-3afd7d46ed1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:45.304541 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415fb27-d316-41ca-8066-3afd7d46ed1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:45.315525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415fb27-d316-41ca-8066-3afd7d46ed1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:45.317006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415fb27-d316-41ca-8066-3afd7d46ed1b service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:45.328612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 956/1906] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:45.402542 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941225a3-1b70-4dea-b11a-4d507739c88e None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:45.414598 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415fb27-d316-41ca-8066-3afd7d46ed1b service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:45.435246 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415fb27-d316-41ca-8066-3afd7d46ed1b service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:45.442566 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfae96-22ad-4baf-9931-2b7b98e22813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:45.445418 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfae96-22ad-4baf-9931-2b7b98e22813 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:45.446059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfae96-22ad-4baf-9931-2b7b98e22813 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:45.447237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfae96-22ad-4baf-9931-2b7b98e22813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:45.546232 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941225a3-1b70-4dea-b11a-4d507739c88e None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:45.554367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd7705-cbd6-4517-b2d0-5b49d5ca3979 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0be06ca30b694345930bda307adce08d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d34fcca57cd4fa49363f76a36c00ffe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:45.555576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd7705-cbd6-4517-b2d0-5b49d5ca3979 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:45.555576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd7705-cbd6-4517-b2d0-5b49d5ca3979 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:45.555777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd7705-cbd6-4517-b2d0-5b49d5ca3979 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:45.556657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd7705-cbd6-4517-b2d0-5b49d5ca3979 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:45.557151 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd7705-cbd6-4517-b2d0-5b49d5ca3979 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:45.559430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd7705-cbd6-4517-b2d0-5b49d5ca3979 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:45.600434 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 951/1907] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:45.604574 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd7705-cbd6-4517-b2d0-5b49d5ca3979 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:45.605042 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfae96-22ad-4baf-9931-2b7b98e22813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:45.607333 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfae96-22ad-4baf-9931-2b7b98e22813 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:45.620902 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd7705-cbd6-4517-b2d0-5b49d5ca3979 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:45.623846 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 957/1908] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:45.632247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87d61b-4a15-4d67-8715-c05a4f22f6ad tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0be06ca30b694345930bda307adce08d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d34fcca57cd4fa49363f76a36c00ffe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:45.633534 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87d61b-4a15-4d67-8715-c05a4f22f6ad tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:45.634539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87d61b-4a15-4d67-8715-c05a4f22f6ad tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:45.634913 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87d61b-4a15-4d67-8715-c05a4f22f6ad tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:45.636273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87d61b-4a15-4d67-8715-c05a4f22f6ad tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:45.638966 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87d61b-4a15-4d67-8715-c05a4f22f6ad tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:45.676702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:45.695944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 952/1909] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:45 2022] POST /identity/v3/projects => generated 352 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:45.700071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfae96-22ad-4baf-9931-2b7b98e22813 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:45.716327 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfae96-22ad-4baf-9931-2b7b98e22813 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:45.817707 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:45.835709 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca353a-9a37-432f-86d3-d8620776b468 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0be06ca30b694345930bda307adce08d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d34fcca57cd4fa49363f76a36c00ffe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:45.836525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca353a-9a37-432f-86d3-d8620776b468 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:45.836831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca353a-9a37-432f-86d3-d8620776b468 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:45.837187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca353a-9a37-432f-86d3-d8620776b468 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:45.837975 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca353a-9a37-432f-86d3-d8620776b468 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:45.838421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca353a-9a37-432f-86d3-d8620776b468 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:45.840337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca353a-9a37-432f-86d3-d8620776b468 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:45.853963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 958/1910] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:45.909813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87d61b-4a15-4d67-8715-c05a4f22f6ad tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:45.926228 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87d61b-4a15-4d67-8715-c05a4f22f6ad tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:45.945488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931b7dc-698f-4c98-bcd0-e0b8539b80ed tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0be06ca30b694345930bda307adce08d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d34fcca57cd4fa49363f76a36c00ffe', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:45.946539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931b7dc-698f-4c98-bcd0-e0b8539b80ed tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:45.946834 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931b7dc-698f-4c98-bcd0-e0b8539b80ed tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:45.947118 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931b7dc-698f-4c98-bcd0-e0b8539b80ed tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] PATH_INFO: `/v3/projects/ef7705b6acf4445ea04ee0affe1b37e4` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:45.948546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931b7dc-698f-4c98-bcd0-e0b8539b80ed tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ef7705b6acf4445ea04ee0affe1b37e4)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:45.950469 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931b7dc-698f-4c98-bcd0-e0b8539b80ed tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:46.070163 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a931b7dc-698f-4c98-bcd0-e0b8539b80ed tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef7705b6acf4445ea04ee0affe1b37e4', 'request_id': 'req-a931b7dc-698f-4c98-bcd0-e0b8539b80ed'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:46.089556 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a931b7dc-698f-4c98-bcd0-e0b8539b80ed tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef7705b6acf4445ea04ee0affe1b37e4', 'request_id': 'req-a931b7dc-698f-4c98-bcd0-e0b8539b80ed'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:46.124340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a931b7dc-698f-4c98-bcd0-e0b8539b80ed tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef7705b6acf4445ea04ee0affe1b37e4', 'request_id': 'req-a931b7dc-698f-4c98-bcd0-e0b8539b80ed'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:46.158264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a931b7dc-698f-4c98-bcd0-e0b8539b80ed tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] The token cache is being invalidate because project ef7705b6acf4445ea04ee0affe1b37e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:46.158565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a931b7dc-698f-4c98-bcd0-e0b8539b80ed tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:46.159609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a931b7dc-698f-4c98-bcd0-e0b8539b80ed tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:46.162339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 953/1911] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:45 2022] DELETE /identity/v3/projects/ef7705b6acf4445ea04ee0affe1b37e4 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:46.398018 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca353a-9a37-432f-86d3-d8620776b468 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:46.400930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-44ca353a-9a37-432f-86d3-d8620776b468 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:46.485227 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca353a-9a37-432f-86d3-d8620776b468 tempest-VolumeQuotaClassesTest-1616227098 tempest-VolumeQuotaClassesTest-1616227098-project-admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:46.549174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425f25a-3d83-4e9b-b3d8-8ca90f83a6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:46.550054 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425f25a-3d83-4e9b-b3d8-8ca90f83a6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:46.550345 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425f25a-3d83-4e9b-b3d8-8ca90f83a6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:46.550627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425f25a-3d83-4e9b-b3d8-8ca90f83a6ab admin admin] PATH_INFO: `/v3/users/ca6dac26fbce4f6aae29633a0bcf4d87` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:46.551835 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425f25a-3d83-4e9b-b3d8-8ca90f83a6ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca6dac26fbce4f6aae29633a0bcf4d87)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:46.553848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425f25a-3d83-4e9b-b3d8-8ca90f83a6ab admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:46.613893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3425f25a-3d83-4e9b-b3d8-8ca90f83a6ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca6dac26fbce4f6aae29633a0bcf4d87'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:46.623680 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3425f25a-3d83-4e9b-b3d8-8ca90f83a6ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca6dac26fbce4f6aae29633a0bcf4d87'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:46.624522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3425f25a-3d83-4e9b-b3d8-8ca90f83a6ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca6dac26fbce4f6aae29633a0bcf4d87'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:46.633786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3425f25a-3d83-4e9b-b3d8-8ca90f83a6ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca6dac26fbce4f6aae29633a0bcf4d87'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:46.655041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3425f25a-3d83-4e9b-b3d8-8ca90f83a6ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca6dac26fbce4f6aae29633a0bcf4d87'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:46.662513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 959/1912] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:46 2022] DELETE /identity/v3/users/ca6dac26fbce4f6aae29633a0bcf4d87 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:46.666712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f06d9f-fdcf-4c94-9215-f0cf60dfad77 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:46.667305 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f06d9f-fdcf-4c94-9215-f0cf60dfad77 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:46.667698 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f06d9f-fdcf-4c94-9215-f0cf60dfad77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:46.710419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-c8f06d9f-fdcf-4c94-9215-f0cf60dfad77 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:46.711713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c8f06d9f-fdcf-4c94-9215-f0cf60dfad77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:46.790286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 954/1913] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:46.803233 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425f25a-3d83-4e9b-b3d8-8ca90f83a6ab admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:46.805728 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3425f25a-3d83-4e9b-b3d8-8ca90f83a6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:46.820233 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:46.883798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425f25a-3d83-4e9b-b3d8-8ca90f83a6ab admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:46.887828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7777c-789c-4ce5-b01e-2d922b1ad10c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:46.888839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7777c-789c-4ce5-b01e-2d922b1ad10c service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:46.896794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7777c-789c-4ce5-b01e-2d922b1ad10c service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:46.896794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7777c-789c-4ce5-b01e-2d922b1ad10c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:46.905059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7777c-789c-4ce5-b01e-2d922b1ad10c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:46.906574 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7777c-789c-4ce5-b01e-2d922b1ad10c service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:46.916628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 960/1914] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:47.459737 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f06d9f-fdcf-4c94-9215-f0cf60dfad77 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:47.461728 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c8f06d9f-fdcf-4c94-9215-f0cf60dfad77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:47.493098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f06d9f-fdcf-4c94-9215-f0cf60dfad77 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:47.540935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:47.541828 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:47.542256 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:47.542577 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c admin admin] PATH_INFO: `/v3/projects/852ef6f37fd14eb4845270afb2328e32` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:47.544348 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=852ef6f37fd14eb4845270afb2328e32)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:47.546432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:47.622718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '852ef6f37fd14eb4845270afb2328e32', 'request_id': 'req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:47.640604 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '852ef6f37fd14eb4845270afb2328e32', 'request_id': 'req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:47.653487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '852ef6f37fd14eb4845270afb2328e32', 'request_id': 'req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:47.685445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c admin admin] The token cache is being invalidate because project 852ef6f37fd14eb4845270afb2328e32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:47.685880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:47.687444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:47.691751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 955/1915] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:47 2022] DELETE /identity/v3/projects/852ef6f37fd14eb4845270afb2328e32 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:47.696094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7777c-789c-4ce5-b01e-2d922b1ad10c service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:47.711463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7777c-789c-4ce5-b01e-2d922b1ad10c service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:47.801919 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb82d74-00ed-4c2a-8a10-afccc96be4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:47.802848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb82d74-00ed-4c2a-8a10-afccc96be4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:47.803199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb82d74-00ed-4c2a-8a10-afccc96be4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:47.803567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb82d74-00ed-4c2a-8a10-afccc96be4d8 admin admin] PATH_INFO: `/v3/users/0be06ca30b694345930bda307adce08d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:47.804746 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb82d74-00ed-4c2a-8a10-afccc96be4d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0be06ca30b694345930bda307adce08d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:47.806671 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb82d74-00ed-4c2a-8a10-afccc96be4d8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:47.906642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4bb82d74-00ed-4c2a-8a10-afccc96be4d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0be06ca30b694345930bda307adce08d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:47.928198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4bb82d74-00ed-4c2a-8a10-afccc96be4d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0be06ca30b694345930bda307adce08d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:47.928966 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4bb82d74-00ed-4c2a-8a10-afccc96be4d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0be06ca30b694345930bda307adce08d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:47.941709 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4bb82d74-00ed-4c2a-8a10-afccc96be4d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0be06ca30b694345930bda307adce08d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:47.968370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4bb82d74-00ed-4c2a-8a10-afccc96be4d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0be06ca30b694345930bda307adce08d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:47.985131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 961/1916] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:47 2022] DELETE /identity/v3/users/0be06ca30b694345930bda307adce08d => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:48.625099 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:48.626753 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:48.650942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f149ef-b3f4-4da1-a53f-4d5806f72d8c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:48.655082 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:48.722596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78339c-4fb9-4933-83a6-70808c780897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:48.723522 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78339c-4fb9-4933-83a6-70808c780897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:48.723804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78339c-4fb9-4933-83a6-70808c780897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:48.724098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78339c-4fb9-4933-83a6-70808c780897 admin admin] PATH_INFO: `/v3/projects/3d34fcca57cd4fa49363f76a36c00ffe` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:48.725343 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f78339c-4fb9-4933-83a6-70808c780897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d34fcca57cd4fa49363f76a36c00ffe)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:48.727226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f78339c-4fb9-4933-83a6-70808c780897 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:48.775482 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4f78339c-4fb9-4933-83a6-70808c780897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d34fcca57cd4fa49363f76a36c00ffe', 'request_id': 'req-4f78339c-4fb9-4933-83a6-70808c780897'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:48.782238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4f78339c-4fb9-4933-83a6-70808c780897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d34fcca57cd4fa49363f76a36c00ffe', 'request_id': 'req-4f78339c-4fb9-4933-83a6-70808c780897'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:48.788408 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4f78339c-4fb9-4933-83a6-70808c780897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d34fcca57cd4fa49363f76a36c00ffe', 'request_id': 'req-4f78339c-4fb9-4933-83a6-70808c780897'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:48.800588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4f78339c-4fb9-4933-83a6-70808c780897 admin admin] The token cache is being invalidate because project 3d34fcca57cd4fa49363f76a36c00ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:48.800876 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4f78339c-4fb9-4933-83a6-70808c780897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:48.801843 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4f78339c-4fb9-4933-83a6-70808c780897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:48.803415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 956/1917] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:48 2022] DELETE /identity/v3/projects/3d34fcca57cd4fa49363f76a36c00ffe => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:48.808612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215e92a-fc50-485f-8926-d215bd3ab10e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:48.809069 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215e92a-fc50-485f-8926-d215bd3ab10e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:48.809156 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215e92a-fc50-485f-8926-d215bd3ab10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:48.827784 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-9215e92a-fc50-485f-8926-d215bd3ab10e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:48.829193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9215e92a-fc50-485f-8926-d215bd3ab10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:48.858830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:48.897933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 962/1918] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:48.905641 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac7c24-646f-4645-8afc-823cc2b19141 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:48.905940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac7c24-646f-4645-8afc-823cc2b19141 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:48.906219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac7c24-646f-4645-8afc-823cc2b19141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:48.924144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-29ac7c24-646f-4645-8afc-823cc2b19141 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:48.925364 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-29ac7c24-646f-4645-8afc-823cc2b19141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:48.933467 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 957/1919] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:48.937814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215e92a-fc50-485f-8926-d215bd3ab10e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:48.948047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215e92a-fc50-485f-8926-d215bd3ab10e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:48.952578 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc90fd-93c3-430b-b842-db52cd242248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:48.953226 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc90fd-93c3-430b-b842-db52cd242248 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:48.953510 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc90fd-93c3-430b-b842-db52cd242248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:48.953788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc90fd-93c3-430b-b842-db52cd242248 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:48.954519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc90fd-93c3-430b-b842-db52cd242248 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:48.954829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc90fd-93c3-430b-b842-db52cd242248 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:48.956453 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc90fd-93c3-430b-b842-db52cd242248 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:48.972412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 963/1920] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:48.973489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac7c24-646f-4645-8afc-823cc2b19141 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:48.984935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac7c24-646f-4645-8afc-823cc2b19141 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:48.990562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6030dd-2008-4200-984b-84e95b35939d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:48.991221 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6030dd-2008-4200-984b-84e95b35939d admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:48.991506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6030dd-2008-4200-984b-84e95b35939d admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:48.991780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6030dd-2008-4200-984b-84e95b35939d admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:48.992523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6030dd-2008-4200-984b-84e95b35939d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:48.994395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6030dd-2008-4200-984b-84e95b35939d admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:49.021881 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 958/1921] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:48 2022] POST /identity/v3/projects => generated 361 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:49.026134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc90fd-93c3-430b-b842-db52cd242248 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:49.062232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc90fd-93c3-430b-b842-db52cd242248 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:49.126685 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c6d8f-fd86-49f7-bbf8-e672992a4e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:49.127140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c6d8f-fd86-49f7-bbf8-e672992a4e04 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.127421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c6d8f-fd86-49f7-bbf8-e672992a4e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.127714 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c6d8f-fd86-49f7-bbf8-e672992a4e04 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:49.128544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c6d8f-fd86-49f7-bbf8-e672992a4e04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:49.130576 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c6d8f-fd86-49f7-bbf8-e672992a4e04 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:49.143700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6030dd-2008-4200-984b-84e95b35939d admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:49.145738 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6d6030dd-2008-4200-984b-84e95b35939d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:49.154650 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 964/1922] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:49 2022] POST /identity/v3/users => generated 281 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:49.158792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c6d8f-fd86-49f7-bbf8-e672992a4e04 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:49.163310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6030dd-2008-4200-984b-84e95b35939d admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:49.168902 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d893d-a410-46c3-9b35-ee00643c1108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:49.169240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c6d8f-fd86-49f7-bbf8-e672992a4e04 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:49.169855 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d893d-a410-46c3-9b35-ee00643c1108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.170142 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d893d-a410-46c3-9b35-ee00643c1108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.170420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d893d-a410-46c3-9b35-ee00643c1108 admin admin] PATH_INFO: `/v3/users/ac3ee12752604c0eb49846f07ec920d9` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:49.171900 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d893d-a410-46c3-9b35-ee00643c1108 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac3ee12752604c0eb49846f07ec920d9)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:49.173854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d893d-a410-46c3-9b35-ee00643c1108 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:49.175715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb9d8b-2aa9-498f-9b13-9ba1a8c6817a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:49.176455 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb9d8b-2aa9-498f-9b13-9ba1a8c6817a admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.176742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb9d8b-2aa9-498f-9b13-9ba1a8c6817a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.177128 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb9d8b-2aa9-498f-9b13-9ba1a8c6817a admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:49.178552 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb9d8b-2aa9-498f-9b13-9ba1a8c6817a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:49.180129 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb9d8b-2aa9-498f-9b13-9ba1a8c6817a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:49.196016 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 965/1923] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:49.197360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb9d8b-2aa9-498f-9b13-9ba1a8c6817a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:49.207972 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb9d8b-2aa9-498f-9b13-9ba1a8c6817a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:49.213758 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef53fd8-bba9-408d-a476-8fbbdb0e60d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:49.214470 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef53fd8-bba9-408d-a476-8fbbdb0e60d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.214775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef53fd8-bba9-408d-a476-8fbbdb0e60d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.215156 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef53fd8-bba9-408d-a476-8fbbdb0e60d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:49.215867 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef53fd8-bba9-408d-a476-8fbbdb0e60d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:49.218080 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef53fd8-bba9-408d-a476-8fbbdb0e60d3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:49.232770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 966/1924] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:49.233759 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef53fd8-bba9-408d-a476-8fbbdb0e60d3 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:49.239678 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e36d893d-a410-46c3-9b35-ee00643c1108 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac3ee12752604c0eb49846f07ec920d9'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.244961 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef53fd8-bba9-408d-a476-8fbbdb0e60d3 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:49.250087 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e36d893d-a410-46c3-9b35-ee00643c1108 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac3ee12752604c0eb49846f07ec920d9'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.250526 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44caa317-7c41-4573-bbe7-67d5349b98f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:49.253368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caa317-7c41-4573-bbe7-67d5349b98f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.253654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caa317-7c41-4573-bbe7-67d5349b98f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.253933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caa317-7c41-4573-bbe7-67d5349b98f6 admin admin] PATH_INFO: `/v3/projects/b3fa280d632d49a9b65d1da9732d7001/users/03d1e414d6314f6da61d234b3c0475e2/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:49.259715 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:49.268609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e36d893d-a410-46c3-9b35-ee00643c1108 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac3ee12752604c0eb49846f07ec920d9'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.271859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44caa317-7c41-4573-bbe7-67d5349b98f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3fa280d632d49a9b65d1da9732d7001, user_id=03d1e414d6314f6da61d234b3c0475e2, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:49.274588 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44caa317-7c41-4573-bbe7-67d5349b98f6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:49.278332 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e36d893d-a410-46c3-9b35-ee00643c1108 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac3ee12752604c0eb49846f07ec920d9'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.279266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e36d893d-a410-46c3-9b35-ee00643c1108 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac3ee12752604c0eb49846f07ec920d9'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.286930 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 959/1925] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:49 2022] DELETE /identity/v3/users/ac3ee12752604c0eb49846f07ec920d9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:49.290974 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7493e7b-b3c8-48ba-a9f9-3ce3bce71598 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.291269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7493e7b-b3c8-48ba-a9f9-3ce3bce71598 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.299774 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7493e7b-b3c8-48ba-a9f9-3ce3bce71598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:49.299941 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 967/1926] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:49 2022] PUT /identity/v3/projects/b3fa280d632d49a9b65d1da9732d7001/users/03d1e414d6314f6da61d234b3c0475e2/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:49.301918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ec5ff-506a-479e-96a0-d6484d97765d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.302260 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ec5ff-506a-479e-96a0-d6484d97765d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.303019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ec5ff-506a-479e-96a0-d6484d97765d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:49.317423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-b7493e7b-b3c8-48ba-a9f9-3ce3bce71598 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:49.319213 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-b7493e7b-b3c8-48ba-a9f9-3ce3bce71598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:49.330123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-dc9ec5ff-506a-479e-96a0-d6484d97765d None None] MFA Rules not processed for user `03d1e414d6314f6da61d234b3c0475e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:49.331319 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-dc9ec5ff-506a-479e-96a0-d6484d97765d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:49.389998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:49.405764 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 960/1927] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:49.415818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7493e7b-b3c8-48ba-a9f9-3ce3bce71598 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:49.418175 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-b7493e7b-b3c8-48ba-a9f9-3ce3bce71598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:49.419964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 968/1928] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:49.453340 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ec5ff-506a-479e-96a0-d6484d97765d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:49.454727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-dc9ec5ff-506a-479e-96a0-d6484d97765d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:49.488917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7493e7b-b3c8-48ba-a9f9-3ce3bce71598 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:49.495089 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6824da-7c67-45d2-bff7-d0487d13ff36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:49.495917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6824da-7c67-45d2-bff7-d0487d13ff36 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.496204 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6824da-7c67-45d2-bff7-d0487d13ff36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.496693 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6824da-7c67-45d2-bff7-d0487d13ff36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:49.513651 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6824da-7c67-45d2-bff7-d0487d13ff36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:49.515701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6824da-7c67-45d2-bff7-d0487d13ff36 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:49.529226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 961/1929] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:49.540032 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ec5ff-506a-479e-96a0-d6484d97765d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:49.544101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7858d7-63b0-4017-ad2c-60c49c300def service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:49.544869 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7858d7-63b0-4017-ad2c-60c49c300def service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.545083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7858d7-63b0-4017-ad2c-60c49c300def service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.545349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7858d7-63b0-4017-ad2c-60c49c300def service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:49.557579 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7858d7-63b0-4017-ad2c-60c49c300def service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:49.559067 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7858d7-63b0-4017-ad2c-60c49c300def service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:49.570955 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 969/1930] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:49.586100 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6824da-7c67-45d2-bff7-d0487d13ff36 service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:49.588647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ca6824da-7c67-45d2-bff7-d0487d13ff36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:49.602707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6824da-7c67-45d2-bff7-d0487d13ff36 service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:49.608114 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b014f-8f86-41b8-8193-d36f688ac42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:49.609051 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b014f-8f86-41b8-8193-d36f688ac42f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.609335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b014f-8f86-41b8-8193-d36f688ac42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.609615 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b014f-8f86-41b8-8193-d36f688ac42f admin admin] PATH_INFO: `/v3/users/03d1e414d6314f6da61d234b3c0475e2` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:49.610863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b014f-8f86-41b8-8193-d36f688ac42f admin admin] RBAC: Authorizing `identity:delete_user(user_id=03d1e414d6314f6da61d234b3c0475e2)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:49.613089 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b014f-8f86-41b8-8193-d36f688ac42f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:49.672840 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-800b014f-8f86-41b8-8193-d36f688ac42f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03d1e414d6314f6da61d234b3c0475e2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.682756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-800b014f-8f86-41b8-8193-d36f688ac42f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03d1e414d6314f6da61d234b3c0475e2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.701844 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-800b014f-8f86-41b8-8193-d36f688ac42f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03d1e414d6314f6da61d234b3c0475e2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.708018 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:49.711090 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-800b014f-8f86-41b8-8193-d36f688ac42f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03d1e414d6314f6da61d234b3c0475e2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.714877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-800b014f-8f86-41b8-8193-d36f688ac42f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03d1e414d6314f6da61d234b3c0475e2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.729041 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 962/1931] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:49 2022] DELETE /identity/v3/users/03d1e414d6314f6da61d234b3c0475e2 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:49.733351 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296f773-9443-46b5-8e8a-27fb48bb75e9 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.733654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296f773-9443-46b5-8e8a-27fb48bb75e9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.733948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296f773-9443-46b5-8e8a-27fb48bb75e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:49.764521 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f296f773-9443-46b5-8e8a-27fb48bb75e9 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:49.766081 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f296f773-9443-46b5-8e8a-27fb48bb75e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:49.896169 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 970/1932] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:49.905499 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b014f-8f86-41b8-8193-d36f688ac42f admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:49.907246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-800b014f-8f86-41b8-8193-d36f688ac42f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:49.944852 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296f773-9443-46b5-8e8a-27fb48bb75e9 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:49.946845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f296f773-9443-46b5-8e8a-27fb48bb75e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:49.959336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296f773-9443-46b5-8e8a-27fb48bb75e9 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:49.964134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a913e6-748d-4805-8e2a-566720c55ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:49.965243 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a913e6-748d-4805-8e2a-566720c55ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.965582 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a913e6-748d-4805-8e2a-566720c55ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.966130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a913e6-748d-4805-8e2a-566720c55ec4 admin admin] PATH_INFO: `/v3/users/573f2b6df5c24856beedc84afcd92fbf` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:49.967565 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a913e6-748d-4805-8e2a-566720c55ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=573f2b6df5c24856beedc84afcd92fbf)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:49.971046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a913e6-748d-4805-8e2a-566720c55ec4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:49.975417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b014f-8f86-41b8-8193-d36f688ac42f admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:49.984048 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4f9c5-7f01-4c25-8580-c37b539b4689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:49.985135 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4f9c5-7f01-4c25-8580-c37b539b4689 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.985607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4f9c5-7f01-4c25-8580-c37b539b4689 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.986318 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4f9c5-7f01-4c25-8580-c37b539b4689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.001686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4f9c5-7f01-4c25-8580-c37b539b4689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:50.003398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4f9c5-7f01-4c25-8580-c37b539b4689 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:50.018936 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 963/1933] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:50.019824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4f9c5-7f01-4c25-8580-c37b539b4689 service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:50.021957 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-00e4f9c5-7f01-4c25-8580-c37b539b4689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:50.038220 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4f9c5-7f01-4c25-8580-c37b539b4689 service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:50.040952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-53a913e6-748d-4805-8e2a-566720c55ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '573f2b6df5c24856beedc84afcd92fbf'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.045237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce631f4b-9e8e-49ff-bd42-6105141feaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:50.046108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce631f4b-9e8e-49ff-bd42-6105141feaa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.046390 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce631f4b-9e8e-49ff-bd42-6105141feaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.046668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce631f4b-9e8e-49ff-bd42-6105141feaa9 admin admin] PATH_INFO: `/v3/projects/9700a04efbbc4c3fbaac0fc5ed2e091f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.048007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce631f4b-9e8e-49ff-bd42-6105141feaa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9700a04efbbc4c3fbaac0fc5ed2e091f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:50.050296 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce631f4b-9e8e-49ff-bd42-6105141feaa9 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:50.058606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-53a913e6-748d-4805-8e2a-566720c55ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '573f2b6df5c24856beedc84afcd92fbf'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.064111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-53a913e6-748d-4805-8e2a-566720c55ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '573f2b6df5c24856beedc84afcd92fbf'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.074827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-53a913e6-748d-4805-8e2a-566720c55ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '573f2b6df5c24856beedc84afcd92fbf'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.101103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-53a913e6-748d-4805-8e2a-566720c55ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '573f2b6df5c24856beedc84afcd92fbf'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.112823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 971/1934] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:49 2022] DELETE /identity/v3/users/573f2b6df5c24856beedc84afcd92fbf => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:50.115199 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ce631f4b-9e8e-49ff-bd42-6105141feaa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9700a04efbbc4c3fbaac0fc5ed2e091f', 'request_id': 'req-ce631f4b-9e8e-49ff-bd42-6105141feaa9'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.117837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c3024-4934-4ea9-a086-60a7db37f506 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.119003 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c3024-4934-4ea9-a086-60a7db37f506 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.119516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c3024-4934-4ea9-a086-60a7db37f506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.125582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ce631f4b-9e8e-49ff-bd42-6105141feaa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9700a04efbbc4c3fbaac0fc5ed2e091f', 'request_id': 'req-ce631f4b-9e8e-49ff-bd42-6105141feaa9'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.136035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ce631f4b-9e8e-49ff-bd42-6105141feaa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9700a04efbbc4c3fbaac0fc5ed2e091f', 'request_id': 'req-ce631f4b-9e8e-49ff-bd42-6105141feaa9'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.146955 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:50.153720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ce631f4b-9e8e-49ff-bd42-6105141feaa9 admin admin] The token cache is being invalidate because project 9700a04efbbc4c3fbaac0fc5ed2e091f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:50.154011 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ce631f4b-9e8e-49ff-bd42-6105141feaa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.155168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ce631f4b-9e8e-49ff-bd42-6105141feaa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.157772 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 964/1935] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:50 2022] DELETE /identity/v3/projects/9700a04efbbc4c3fbaac0fc5ed2e091f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:50.158841 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-791c3024-4934-4ea9-a086-60a7db37f506 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:50.161409 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce631f4b-9e8e-49ff-bd42-6105141feaa9 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:50.162387 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-791c3024-4934-4ea9-a086-60a7db37f506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:50.166640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ce631f4b-9e8e-49ff-bd42-6105141feaa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:50.237061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 972/1936] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:50.247873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c3024-4934-4ea9-a086-60a7db37f506 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:50.250087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-791c3024-4934-4ea9-a086-60a7db37f506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:50.256704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce631f4b-9e8e-49ff-bd42-6105141feaa9 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:50.261437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36fc0e-6533-4e72-8342-136d1492c808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:50.262492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36fc0e-6533-4e72-8342-136d1492c808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.262780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36fc0e-6533-4e72-8342-136d1492c808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.263059 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36fc0e-6533-4e72-8342-136d1492c808 admin admin] PATH_INFO: `/v3/users/d137e95588d14319964e1e8578c2954a` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.264449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36fc0e-6533-4e72-8342-136d1492c808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d137e95588d14319964e1e8578c2954a)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:50.266494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36fc0e-6533-4e72-8342-136d1492c808 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:50.306804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:50.342537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c3024-4934-4ea9-a086-60a7db37f506 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:50.350054 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038024b1-eb2f-4dea-a8a2-a36081fc0dfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:50.350762 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038024b1-eb2f-4dea-a8a2-a36081fc0dfd service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.351007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038024b1-eb2f-4dea-a8a2-a36081fc0dfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.351894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038024b1-eb2f-4dea-a8a2-a36081fc0dfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.359204 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-bb36fc0e-6533-4e72-8342-136d1492c808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd137e95588d14319964e1e8578c2954a'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.369610 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038024b1-eb2f-4dea-a8a2-a36081fc0dfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:50.371341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038024b1-eb2f-4dea-a8a2-a36081fc0dfd service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:50.375189 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-bb36fc0e-6533-4e72-8342-136d1492c808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd137e95588d14319964e1e8578c2954a'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.390447 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-bb36fc0e-6533-4e72-8342-136d1492c808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd137e95588d14319964e1e8578c2954a'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.403621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-bb36fc0e-6533-4e72-8342-136d1492c808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd137e95588d14319964e1e8578c2954a'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.403989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 973/1937] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:50.404612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-bb36fc0e-6533-4e72-8342-136d1492c808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd137e95588d14319964e1e8578c2954a'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.411287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038024b1-eb2f-4dea-a8a2-a36081fc0dfd service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:50.412119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 965/1938] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:50 2022] DELETE /identity/v3/users/d137e95588d14319964e1e8578c2954a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:50.415666 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-038024b1-eb2f-4dea-a8a2-a36081fc0dfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:50.429888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038024b1-eb2f-4dea-a8a2-a36081fc0dfd service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:50.528328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b48adf-0f9a-4ada-87f1-8695d17589c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:50.529331 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b48adf-0f9a-4ada-87f1-8695d17589c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.529892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b48adf-0f9a-4ada-87f1-8695d17589c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.530028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b48adf-0f9a-4ada-87f1-8695d17589c6 admin admin] PATH_INFO: `/v3/projects/b3fa280d632d49a9b65d1da9732d7001` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.531180 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b48adf-0f9a-4ada-87f1-8695d17589c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3fa280d632d49a9b65d1da9732d7001)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:50.533513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b48adf-0f9a-4ada-87f1-8695d17589c6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:50.615897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e7b48adf-0f9a-4ada-87f1-8695d17589c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3fa280d632d49a9b65d1da9732d7001', 'request_id': 'req-e7b48adf-0f9a-4ada-87f1-8695d17589c6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.626465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:50.641349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e7b48adf-0f9a-4ada-87f1-8695d17589c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3fa280d632d49a9b65d1da9732d7001', 'request_id': 'req-e7b48adf-0f9a-4ada-87f1-8695d17589c6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.668715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e7b48adf-0f9a-4ada-87f1-8695d17589c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3fa280d632d49a9b65d1da9732d7001', 'request_id': 'req-e7b48adf-0f9a-4ada-87f1-8695d17589c6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.691868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e7b48adf-0f9a-4ada-87f1-8695d17589c6 admin admin] The token cache is being invalidate because project b3fa280d632d49a9b65d1da9732d7001 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:50.692166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e7b48adf-0f9a-4ada-87f1-8695d17589c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.692897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e7b48adf-0f9a-4ada-87f1-8695d17589c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:50.696418 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 974/1939] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:50 2022] DELETE /identity/v3/projects/b3fa280d632d49a9b65d1da9732d7001 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:50.701178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551596f4-6aed-4db8-bf58-76fec8286468 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.701721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551596f4-6aed-4db8-bf58-76fec8286468 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.701721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551596f4-6aed-4db8-bf58-76fec8286468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.771875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-551596f4-6aed-4db8-bf58-76fec8286468 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:50.777878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-551596f4-6aed-4db8-bf58-76fec8286468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:50.932049 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 966/1940] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:50.942017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283e1f5-6b3b-44d2-a2d8-669f5aeca686 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.942473 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283e1f5-6b3b-44d2-a2d8-669f5aeca686 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.942619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283e1f5-6b3b-44d2-a2d8-669f5aeca686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.975571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5283e1f5-6b3b-44d2-a2d8-669f5aeca686 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:50.976849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5283e1f5-6b3b-44d2-a2d8-669f5aeca686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:51.003193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 975/1941] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:51.005949 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551596f4-6aed-4db8-bf58-76fec8286468 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:51.017012 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551596f4-6aed-4db8-bf58-76fec8286468 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:51.023887 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fb2f2-0c53-4fe2-a999-eb844743f1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:51.024457 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283e1f5-6b3b-44d2-a2d8-669f5aeca686 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:51.025362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fb2f2-0c53-4fe2-a999-eb844743f1ec admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:51.025792 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fb2f2-0c53-4fe2-a999-eb844743f1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:51.025978 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5283e1f5-6b3b-44d2-a2d8-669f5aeca686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:51.026285 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fb2f2-0c53-4fe2-a999-eb844743f1ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:51.027188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fb2f2-0c53-4fe2-a999-eb844743f1ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:51.027545 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fb2f2-0c53-4fe2-a999-eb844743f1ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:51.029405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fb2f2-0c53-4fe2-a999-eb844743f1ec admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:51.046942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283e1f5-6b3b-44d2-a2d8-669f5aeca686 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:51.049757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 967/1942] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:51.054967 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fb2f2-0c53-4fe2-a999-eb844743f1ec admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:51.057939 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258f499-d0a6-4212-a7da-1f29fc8517b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:51.058830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258f499-d0a6-4212-a7da-1f29fc8517b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:51.059118 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258f499-d0a6-4212-a7da-1f29fc8517b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:51.059573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258f499-d0a6-4212-a7da-1f29fc8517b0 admin admin] PATH_INFO: `/v3/projects/0b5883eef00c4f2d925f7d026c13e745` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:51.069258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258f499-d0a6-4212-a7da-1f29fc8517b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b5883eef00c4f2d925f7d026c13e745)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:51.071226 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258f499-d0a6-4212-a7da-1f29fc8517b0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:51.079350 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fb2f2-0c53-4fe2-a999-eb844743f1ec admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:51.094714 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c4b29-ba47-4fcf-8f5c-7f9cabd63aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:51.095467 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c4b29-ba47-4fcf-8f5c-7f9cabd63aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:51.095753 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c4b29-ba47-4fcf-8f5c-7f9cabd63aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:51.096028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c4b29-ba47-4fcf-8f5c-7f9cabd63aa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:51.096844 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c4b29-ba47-4fcf-8f5c-7f9cabd63aa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:51.098815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c4b29-ba47-4fcf-8f5c-7f9cabd63aa5 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:51.146797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 968/1943] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:51 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:51.150026 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c4b29-ba47-4fcf-8f5c-7f9cabd63aa5 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:51.163986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8258f499-d0a6-4212-a7da-1f29fc8517b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b5883eef00c4f2d925f7d026c13e745', 'request_id': 'req-8258f499-d0a6-4212-a7da-1f29fc8517b0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:51.173470 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8258f499-d0a6-4212-a7da-1f29fc8517b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b5883eef00c4f2d925f7d026c13e745', 'request_id': 'req-8258f499-d0a6-4212-a7da-1f29fc8517b0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:51.184527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8258f499-d0a6-4212-a7da-1f29fc8517b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b5883eef00c4f2d925f7d026c13e745', 'request_id': 'req-8258f499-d0a6-4212-a7da-1f29fc8517b0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:51.199582 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8258f499-d0a6-4212-a7da-1f29fc8517b0 admin admin] The token cache is being invalidate because project 0b5883eef00c4f2d925f7d026c13e745 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:51.199935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8258f499-d0a6-4212-a7da-1f29fc8517b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:51.200832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8258f499-d0a6-4212-a7da-1f29fc8517b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:51.202733 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 976/1944] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:51 2022] DELETE /identity/v3/projects/0b5883eef00c4f2d925f7d026c13e745 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:51.206921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258f499-d0a6-4212-a7da-1f29fc8517b0 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:51.209340 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8258f499-d0a6-4212-a7da-1f29fc8517b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:51.255551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c4b29-ba47-4fcf-8f5c-7f9cabd63aa5 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:51.263221 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42aca1f-9c13-4040-8772-2503d8253498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:51.264361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aca1f-9c13-4040-8772-2503d8253498 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:51.264805 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aca1f-9c13-4040-8772-2503d8253498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:51.265233 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aca1f-9c13-4040-8772-2503d8253498 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:51.266345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42aca1f-9c13-4040-8772-2503d8253498 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:51.268851 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42aca1f-9c13-4040-8772-2503d8253498 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:51.282506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:51.299525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258f499-d0a6-4212-a7da-1f29fc8517b0 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:51.308874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 969/1945] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:51 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:51.313500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c4be4f-166e-4d29-985b-b7993099f01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:51.313993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42aca1f-9c13-4040-8772-2503d8253498 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:51.322635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4be4f-166e-4d29-985b-b7993099f01b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:51.323047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4be4f-166e-4d29-985b-b7993099f01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:51.328772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4be4f-166e-4d29-985b-b7993099f01b admin admin] PATH_INFO: `/v3/users/f5a9c3808b2143fdb895ee8a633a7bc6` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:51.334100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c4be4f-166e-4d29-985b-b7993099f01b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5a9c3808b2143fdb895ee8a633a7bc6)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:51.337336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c4be4f-166e-4d29-985b-b7993099f01b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:51.342071 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e42aca1f-9c13-4040-8772-2503d8253498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:51.363532 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42aca1f-9c13-4040-8772-2503d8253498 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:51.377074 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f65816-7ffc-4a57-881f-8bb2a320eb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:51.379447 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f65816-7ffc-4a57-881f-8bb2a320eb14 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:51.379987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f65816-7ffc-4a57-881f-8bb2a320eb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:51.380358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f65816-7ffc-4a57-881f-8bb2a320eb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:51.381292 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f65816-7ffc-4a57-881f-8bb2a320eb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:51.383564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f65816-7ffc-4a57-881f-8bb2a320eb14 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:51.397018 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 970/1946] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:51.401423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f65816-7ffc-4a57-881f-8bb2a320eb14 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:51.416062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f65816-7ffc-4a57-881f-8bb2a320eb14 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:51.422503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18785971-7e7a-421b-8ccc-9bfdae846ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:51.423315 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18785971-7e7a-421b-8ccc-9bfdae846ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:51.423707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18785971-7e7a-421b-8ccc-9bfdae846ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:51.424109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18785971-7e7a-421b-8ccc-9bfdae846ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:51.425498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18785971-7e7a-421b-8ccc-9bfdae846ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:51.427013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18785971-7e7a-421b-8ccc-9bfdae846ea8 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:51.442821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b3c4be4f-166e-4d29-985b-b7993099f01b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5a9c3808b2143fdb895ee8a633a7bc6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:51.454586 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 971/1947] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:51.459830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18785971-7e7a-421b-8ccc-9bfdae846ea8 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:51.466715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b3c4be4f-166e-4d29-985b-b7993099f01b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5a9c3808b2143fdb895ee8a633a7bc6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:51.467682 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b3c4be4f-166e-4d29-985b-b7993099f01b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5a9c3808b2143fdb895ee8a633a7bc6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:51.476154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=79,keystone:INSERT=3,keystone:DELETE=18 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:51.481645 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18785971-7e7a-421b-8ccc-9bfdae846ea8 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:51.483533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b3c4be4f-166e-4d29-985b-b7993099f01b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5a9c3808b2143fdb895ee8a633a7bc6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:51.492696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a776c59-02d8-4445-ad8c-910dcc3ae6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:51.493605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a776c59-02d8-4445-ad8c-910dcc3ae6c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:51.493893 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a776c59-02d8-4445-ad8c-910dcc3ae6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:51.494177 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a776c59-02d8-4445-ad8c-910dcc3ae6c0 admin admin] PATH_INFO: `/v3/projects/dfdff1e88a544e60993c77327684f672/users/59a8d6ff130c4b7b9ee5460fa56dcca5/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:51.518809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b3c4be4f-166e-4d29-985b-b7993099f01b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5a9c3808b2143fdb895ee8a633a7bc6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:51.529542 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a776c59-02d8-4445-ad8c-910dcc3ae6c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfdff1e88a544e60993c77327684f672, user_id=59a8d6ff130c4b7b9ee5460fa56dcca5, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:51.532231 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a776c59-02d8-4445-ad8c-910dcc3ae6c0 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:51.532728 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 977/1948] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:51 2022] DELETE /identity/v3/users/f5a9c3808b2143fdb895ee8a633a7bc6 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:51.533745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c4be4f-166e-4d29-985b-b7993099f01b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:51.535490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b3c4be4f-166e-4d29-985b-b7993099f01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:51.552613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 972/1949] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:51 2022] PUT /identity/v3/projects/dfdff1e88a544e60993c77327684f672/users/59a8d6ff130c4b7b9ee5460fa56dcca5/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:51.562853 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56501ff8-682b-45cf-81eb-b0a5abc64d5a None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:51.563238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56501ff8-682b-45cf-81eb-b0a5abc64d5a None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:51.563625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56501ff8-682b-45cf-81eb-b0a5abc64d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:51.610017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-56501ff8-682b-45cf-81eb-b0a5abc64d5a None None] MFA Rules not processed for user `59a8d6ff130c4b7b9ee5460fa56dcca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:51.611640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-56501ff8-682b-45cf-81eb-b0a5abc64d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:51.634484 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c4be4f-166e-4d29-985b-b7993099f01b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:51.721755 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d614e-2a31-4203-a7e4-4fc27c7d0d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:51.722655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d614e-2a31-4203-a7e4-4fc27c7d0d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:51.722962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d614e-2a31-4203-a7e4-4fc27c7d0d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:51.723225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d614e-2a31-4203-a7e4-4fc27c7d0d68 admin admin] PATH_INFO: `/v3/projects/e5ef5f43829f4a499d5b006b47fa22f8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:51.726646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d614e-2a31-4203-a7e4-4fc27c7d0d68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5ef5f43829f4a499d5b006b47fa22f8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:51.728596 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d614e-2a31-4203-a7e4-4fc27c7d0d68 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:51.732408 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 973/1950] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:51.741589 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56501ff8-682b-45cf-81eb-b0a5abc64d5a None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:51.743528 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-56501ff8-682b-45cf-81eb-b0a5abc64d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:51.819292 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56501ff8-682b-45cf-81eb-b0a5abc64d5a None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:51.821505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5c7d614e-2a31-4203-a7e4-4fc27c7d0d68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5ef5f43829f4a499d5b006b47fa22f8', 'request_id': 'req-5c7d614e-2a31-4203-a7e4-4fc27c7d0d68'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:51.829689 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5c7d614e-2a31-4203-a7e4-4fc27c7d0d68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5ef5f43829f4a499d5b006b47fa22f8', 'request_id': 'req-5c7d614e-2a31-4203-a7e4-4fc27c7d0d68'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:51.843386 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5c7d614e-2a31-4203-a7e4-4fc27c7d0d68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5ef5f43829f4a499d5b006b47fa22f8', 'request_id': 'req-5c7d614e-2a31-4203-a7e4-4fc27c7d0d68'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:51.864160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5c7d614e-2a31-4203-a7e4-4fc27c7d0d68 admin admin] The token cache is being invalidate because project e5ef5f43829f4a499d5b006b47fa22f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:51.864656 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5c7d614e-2a31-4203-a7e4-4fc27c7d0d68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:51.866526 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5c7d614e-2a31-4203-a7e4-4fc27c7d0d68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:51.869533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 978/1951] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:51 2022] DELETE /identity/v3/projects/e5ef5f43829f4a499d5b006b47fa22f8 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:51.874519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78098c8-6c18-4c56-843d-079aaf1f462c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:51.874812 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78098c8-6c18-4c56-843d-079aaf1f462c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:51.875111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78098c8-6c18-4c56-843d-079aaf1f462c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:51.901131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d78098c8-6c18-4c56-843d-079aaf1f462c None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:51.902461 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d78098c8-6c18-4c56-843d-079aaf1f462c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:51.913563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:51.929013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c8caa-cb14-494b-8d61-c6aea4ea650d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:51.929640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c8caa-cb14-494b-8d61-c6aea4ea650d service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:51.930013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c8caa-cb14-494b-8d61-c6aea4ea650d service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:51.930325 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c8caa-cb14-494b-8d61-c6aea4ea650d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:51.932799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4d9c8caa-cb14-494b-8d61-c6aea4ea650d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:51.945384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:51.989935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 979/1952] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:51.991545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a746879-b115-4ab6-8a38-7e81c4111f03 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:51.991893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a746879-b115-4ab6-8a38-7e81c4111f03 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:51.992247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a746879-b115-4ab6-8a38-7e81c4111f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:52.025699 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8a746879-b115-4ab6-8a38-7e81c4111f03 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:52.026878 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8a746879-b115-4ab6-8a38-7e81c4111f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:52.037319 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c8caa-cb14-494b-8d61-c6aea4ea650d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:52.038709 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 980/1953] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:52.040182 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a746879-b115-4ab6-8a38-7e81c4111f03 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:52.041567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8a746879-b115-4ab6-8a38-7e81c4111f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:52.042787 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c8caa-cb14-494b-8d61-c6aea4ea650d service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:52.054564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 974/1954] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:52.055784 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c8caa-cb14-494b-8d61-c6aea4ea650d service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:52.056395 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a746879-b115-4ab6-8a38-7e81c4111f03 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:52.061738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029d462-52ef-4edb-9183-505c4360f25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:52.062876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029d462-52ef-4edb-9183-505c4360f25f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:52.063214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029d462-52ef-4edb-9183-505c4360f25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:52.063567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029d462-52ef-4edb-9183-505c4360f25f admin admin] PATH_INFO: `/v3/projects/6f78dbc95e9c44d5a534da074822f6ab` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:52.078466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029d462-52ef-4edb-9183-505c4360f25f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f78dbc95e9c44d5a534da074822f6ab)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:52.079885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c8caa-cb14-494b-8d61-c6aea4ea650d service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:52.081019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029d462-52ef-4edb-9183-505c4360f25f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:52.096636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929eb3f-3413-466c-969c-7f3eaa9c1638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:52.097468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929eb3f-3413-466c-969c-7f3eaa9c1638 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:52.097911 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929eb3f-3413-466c-969c-7f3eaa9c1638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:52.098236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929eb3f-3413-466c-969c-7f3eaa9c1638 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:52.099273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929eb3f-3413-466c-969c-7f3eaa9c1638 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:52.100872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929eb3f-3413-466c-969c-7f3eaa9c1638 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:52.103013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929eb3f-3413-466c-969c-7f3eaa9c1638 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:52.124942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 975/1955] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:52.135130 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929eb3f-3413-466c-969c-7f3eaa9c1638 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:52.160661 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929eb3f-3413-466c-969c-7f3eaa9c1638 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:52.160754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b029d462-52ef-4edb-9183-505c4360f25f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f78dbc95e9c44d5a534da074822f6ab', 'request_id': 'req-b029d462-52ef-4edb-9183-505c4360f25f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:52.166045 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b029d462-52ef-4edb-9183-505c4360f25f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f78dbc95e9c44d5a534da074822f6ab', 'request_id': 'req-b029d462-52ef-4edb-9183-505c4360f25f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:52.177964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b029d462-52ef-4edb-9183-505c4360f25f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f78dbc95e9c44d5a534da074822f6ab', 'request_id': 'req-b029d462-52ef-4edb-9183-505c4360f25f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:52.214166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b029d462-52ef-4edb-9183-505c4360f25f admin admin] The token cache is being invalidate because project 6f78dbc95e9c44d5a534da074822f6ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:52.214896 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b029d462-52ef-4edb-9183-505c4360f25f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:52.215896 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b029d462-52ef-4edb-9183-505c4360f25f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:52.221021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 981/1956] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:52 2022] DELETE /identity/v3/projects/6f78dbc95e9c44d5a534da074822f6ab => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:52.227884 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc04a7f-b9c8-4fa8-8742-4183be5134a7 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:52.229254 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc04a7f-b9c8-4fa8-8742-4183be5134a7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:52.229563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc04a7f-b9c8-4fa8-8742-4183be5134a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:52.241218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac76733-131f-432c-9d28-721683af738d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:52.242169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac76733-131f-432c-9d28-721683af738d admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:52.242930 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac76733-131f-432c-9d28-721683af738d admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:52.243333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac76733-131f-432c-9d28-721683af738d admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:52.244717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac76733-131f-432c-9d28-721683af738d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:52.246999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac76733-131f-432c-9d28-721683af738d admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:52.277248 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8bc04a7f-b9c8-4fa8-8742-4183be5134a7 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:52.278694 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8bc04a7f-b9c8-4fa8-8742-4183be5134a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:52.299908 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 976/1957] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:52 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:52.311689 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac76733-131f-432c-9d28-721683af738d admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:52.312139 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 982/1958] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:52.320313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102ab85-b3ca-4949-a36a-bbc75bf8d9c2 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:52.320619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102ab85-b3ca-4949-a36a-bbc75bf8d9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:52.320905 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102ab85-b3ca-4949-a36a-bbc75bf8d9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:52.323912 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-dac76733-131f-432c-9d28-721683af738d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:52.375814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3102ab85-b3ca-4949-a36a-bbc75bf8d9c2 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:52.377071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3102ab85-b3ca-4949-a36a-bbc75bf8d9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:52.453632 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac76733-131f-432c-9d28-721683af738d admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:52.464076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9226d4-0119-495a-9f67-bf7ad76db150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:52.464806 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9226d4-0119-495a-9f67-bf7ad76db150 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:52.465093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9226d4-0119-495a-9f67-bf7ad76db150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:52.465435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9226d4-0119-495a-9f67-bf7ad76db150 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:52.466678 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9226d4-0119-495a-9f67-bf7ad76db150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:52.469822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9226d4-0119-495a-9f67-bf7ad76db150 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:52.504414 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 977/1959] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:52 2022] POST /identity/v3/users => generated 294 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:52.506048 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9226d4-0119-495a-9f67-bf7ad76db150 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:52.519621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9226d4-0119-495a-9f67-bf7ad76db150 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:52.554456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170cb800-a5a1-4285-915b-e7db36c5313b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:52.555152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cb800-a5a1-4285-915b-e7db36c5313b admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:52.555440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cb800-a5a1-4285-915b-e7db36c5313b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:52.555721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cb800-a5a1-4285-915b-e7db36c5313b admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:52.560058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170cb800-a5a1-4285-915b-e7db36c5313b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:52.567086 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170cb800-a5a1-4285-915b-e7db36c5313b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:52.581156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:52.599767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 978/1960] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:52.604643 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170cb800-a5a1-4285-915b-e7db36c5313b admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:52.607786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 983/1961] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:52.612183 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102ab85-b3ca-4949-a36a-bbc75bf8d9c2 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:52.641978 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102ab85-b3ca-4949-a36a-bbc75bf8d9c2 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:52.653976 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9d5a1-f808-4a1a-9647-6c99bfd383b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:52.654876 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170cb800-a5a1-4285-915b-e7db36c5313b admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:52.655816 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9d5a1-f808-4a1a-9647-6c99bfd383b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:52.656111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9d5a1-f808-4a1a-9647-6c99bfd383b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:52.657451 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9d5a1-f808-4a1a-9647-6c99bfd383b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:52.658563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9d5a1-f808-4a1a-9647-6c99bfd383b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:52.658882 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9d5a1-f808-4a1a-9647-6c99bfd383b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:52.665493 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9d5a1-f808-4a1a-9647-6c99bfd383b7 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:52.669052 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde1cb5-14c1-4ed8-bd6b-7b84a9fbd290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:52.669817 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde1cb5-14c1-4ed8-bd6b-7b84a9fbd290 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:52.670152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde1cb5-14c1-4ed8-bd6b-7b84a9fbd290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:52.670590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde1cb5-14c1-4ed8-bd6b-7b84a9fbd290 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:52.671202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde1cb5-14c1-4ed8-bd6b-7b84a9fbd290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:52.673864 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde1cb5-14c1-4ed8-bd6b-7b84a9fbd290 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:52.688443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 979/1962] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:52.698608 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde1cb5-14c1-4ed8-bd6b-7b84a9fbd290 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:52.714354 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 984/1963] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:52.716031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9d5a1-f808-4a1a-9647-6c99bfd383b7 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:52.719658 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde1cb5-14c1-4ed8-bd6b-7b84a9fbd290 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:52.734502 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ef39b-f3df-4015-8b97-28d007ef1fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:52.735439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ef39b-f3df-4015-8b97-28d007ef1fba admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:52.735727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ef39b-f3df-4015-8b97-28d007ef1fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:52.736029 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ef39b-f3df-4015-8b97-28d007ef1fba admin admin] PATH_INFO: `/v3/projects/05a141a07ae04d079500a197ec5a8111/users/cf1744f4712942a9b1244f07ed543943/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:52.738219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9d5a1-f808-4a1a-9647-6c99bfd383b7 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:52.754543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90cf2c-45b3-4940-8a08-788f7799a4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:52.756220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90cf2c-45b3-4940-8a08-788f7799a4ef admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:52.756744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90cf2c-45b3-4940-8a08-788f7799a4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:52.757223 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90cf2c-45b3-4940-8a08-788f7799a4ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:52.760727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90cf2c-45b3-4940-8a08-788f7799a4ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:52.763125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90cf2c-45b3-4940-8a08-788f7799a4ef admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:52.774314 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ef39b-f3df-4015-8b97-28d007ef1fba admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a141a07ae04d079500a197ec5a8111, user_id=cf1744f4712942a9b1244f07ed543943, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:52.777025 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ef39b-f3df-4015-8b97-28d007ef1fba admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:52.797234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:52.805592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 980/1964] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:52 2022] PUT /identity/v3/projects/05a141a07ae04d079500a197ec5a8111/users/cf1744f4712942a9b1244f07ed543943/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:52.817015 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c3a8e-909e-451a-ac1b-2ae873474ada None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:52.817458 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c3a8e-909e-451a-ac1b-2ae873474ada None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:52.817847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c3a8e-909e-451a-ac1b-2ae873474ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:52.823634 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 985/1965] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:52 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:52.826884 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90cf2c-45b3-4940-8a08-788f7799a4ef admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:52.850135 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-772c3a8e-909e-451a-ac1b-2ae873474ada None None] MFA Rules not processed for user `cf1744f4712942a9b1244f07ed543943`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:52.851983 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-772c3a8e-909e-451a-ac1b-2ae873474ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:52.953473 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90cf2c-45b3-4940-8a08-788f7799a4ef admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:52.963934 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ccd00-23a6-4d7e-bcd5-bc3cc7e7ecb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:52.964653 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ccd00-23a6-4d7e-bcd5-bc3cc7e7ecb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:52.964939 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ccd00-23a6-4d7e-bcd5-bc3cc7e7ecb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:52.967852 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ccd00-23a6-4d7e-bcd5-bc3cc7e7ecb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:52.968626 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ccd00-23a6-4d7e-bcd5-bc3cc7e7ecb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:52.974676 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ccd00-23a6-4d7e-bcd5-bc3cc7e7ecb9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:52.993266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 981/1966] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:52.994422 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c3a8e-909e-451a-ac1b-2ae873474ada None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:53.011620 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 986/1967] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:52 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:53.015277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ccd00-23a6-4d7e-bcd5-bc3cc7e7ecb9 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:53.017262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c3a8e-909e-451a-ac1b-2ae873474ada None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:53.031710 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ccd00-23a6-4d7e-bcd5-bc3cc7e7ecb9 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:53.035948 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45d051-06c8-4692-a3e8-63f59bbfc11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:53.036607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45d051-06c8-4692-a3e8-63f59bbfc11f admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:53.036999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45d051-06c8-4692-a3e8-63f59bbfc11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:53.037282 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45d051-06c8-4692-a3e8-63f59bbfc11f admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:53.039088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772336eb-0d44-47d9-b93e-144148f977d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:53.040040 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772336eb-0d44-47d9-b93e-144148f977d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:53.040148 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45d051-06c8-4692-a3e8-63f59bbfc11f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:53.040641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772336eb-0d44-47d9-b93e-144148f977d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:53.041101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772336eb-0d44-47d9-b93e-144148f977d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:53.042073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772336eb-0d44-47d9-b93e-144148f977d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:53.043402 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45d051-06c8-4692-a3e8-63f59bbfc11f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:53.045031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772336eb-0d44-47d9-b93e-144148f977d3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:53.062226 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 987/1968] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:53.067112 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772336eb-0d44-47d9-b93e-144148f977d3 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:53.089715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772336eb-0d44-47d9-b93e-144148f977d3 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:53.091830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 982/1969] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:52 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:53.095815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45d051-06c8-4692-a3e8-63f59bbfc11f admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:53.179051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413730f-755f-4bfa-ad1c-c437c741ff30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:53.180038 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413730f-755f-4bfa-ad1c-c437c741ff30 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:53.181036 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413730f-755f-4bfa-ad1c-c437c741ff30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:53.181341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413730f-755f-4bfa-ad1c-c437c741ff30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:53.182780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413730f-755f-4bfa-ad1c-c437c741ff30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:53.184511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413730f-755f-4bfa-ad1c-c437c741ff30 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:53.186508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45d051-06c8-4692-a3e8-63f59bbfc11f admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:53.194871 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3765ff1-4c52-422d-a9ab-e5226064f6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:53.195563 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3765ff1-4c52-422d-a9ab-e5226064f6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:53.195924 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3765ff1-4c52-422d-a9ab-e5226064f6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:53.196213 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3765ff1-4c52-422d-a9ab-e5226064f6b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:53.196962 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3765ff1-4c52-422d-a9ab-e5226064f6b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:53.198570 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 988/1970] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:53.199060 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3765ff1-4c52-422d-a9ab-e5226064f6b7 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:53.203006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413730f-755f-4bfa-ad1c-c437c741ff30 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:53.215893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413730f-755f-4bfa-ad1c-c437c741ff30 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:53.224944 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b31a5e-9a5f-480b-93bf-e3dfcd6045c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:53.226615 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b31a5e-9a5f-480b-93bf-e3dfcd6045c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:53.227144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b31a5e-9a5f-480b-93bf-e3dfcd6045c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:53.227590 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b31a5e-9a5f-480b-93bf-e3dfcd6045c4 admin admin] PATH_INFO: `/v3/projects/ee028974653e4c6488746fa3ace8d15d/users/ce45c850382f4cdaa3a874b2d1d8365f/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:53.245550 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 983/1971] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:53 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:53.249375 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b31a5e-9a5f-480b-93bf-e3dfcd6045c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee028974653e4c6488746fa3ace8d15d, user_id=ce45c850382f4cdaa3a874b2d1d8365f, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:53.249540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3765ff1-4c52-422d-a9ab-e5226064f6b7 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:53.252462 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b31a5e-9a5f-480b-93bf-e3dfcd6045c4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:53.278631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3765ff1-4c52-422d-a9ab-e5226064f6b7 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:53.280887 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 989/1972] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:53 2022] PUT /identity/v3/projects/ee028974653e4c6488746fa3ace8d15d/users/ce45c850382f4cdaa3a874b2d1d8365f/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:53.282867 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d281c79-0be3-4a48-8b3c-43a0517171c5 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:53.283297 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d281c79-0be3-4a48-8b3c-43a0517171c5 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:53.283869 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d281c79-0be3-4a48-8b3c-43a0517171c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:53.327134 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:53.377700 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8d281c79-0be3-4a48-8b3c-43a0517171c5 None None] MFA Rules not processed for user `ce45c850382f4cdaa3a874b2d1d8365f`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:53.380944 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8d281c79-0be3-4a48-8b3c-43a0517171c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:53.435220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:53.454648 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74efc0-ad9b-421d-92c5-b22dde3804dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:53.455313 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74efc0-ad9b-421d-92c5-b22dde3804dc admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:53.455598 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74efc0-ad9b-421d-92c5-b22dde3804dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:53.455927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74efc0-ad9b-421d-92c5-b22dde3804dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:53.460871 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74efc0-ad9b-421d-92c5-b22dde3804dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:53.462611 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74efc0-ad9b-421d-92c5-b22dde3804dc admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:53.476899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 984/1973] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:53.480839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74efc0-ad9b-421d-92c5-b22dde3804dc admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:53.490490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 990/1974] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:53.495357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d281c79-0be3-4a48-8b3c-43a0517171c5 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:53.498766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74efc0-ad9b-421d-92c5-b22dde3804dc admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:53.507932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0876d77-e056-49d7-94a3-333a7da90970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:53.508983 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0876d77-e056-49d7-94a3-333a7da90970 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:53.509345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0876d77-e056-49d7-94a3-333a7da90970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:53.509697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0876d77-e056-49d7-94a3-333a7da90970 admin admin] PATH_INFO: `/v3/projects/5a039d1f112545c6ae7ef7bbc5c972ba/users/5cf91939a71c490b81c88165df09de9e/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:53.510142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d281c79-0be3-4a48-8b3c-43a0517171c5 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:53.517698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b4bcf-2657-41c9-854a-4a2a6903c9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:53.518374 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b4bcf-2657-41c9-854a-4a2a6903c9a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:53.518713 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b4bcf-2657-41c9-854a-4a2a6903c9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:53.519115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b4bcf-2657-41c9-854a-4a2a6903c9a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:53.519917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b4bcf-2657-41c9-854a-4a2a6903c9a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:53.523684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b4bcf-2657-41c9-854a-4a2a6903c9a8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:53.529399 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0876d77-e056-49d7-94a3-333a7da90970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a039d1f112545c6ae7ef7bbc5c972ba, user_id=5cf91939a71c490b81c88165df09de9e, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:53.535941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0876d77-e056-49d7-94a3-333a7da90970 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:53.562221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 991/1975] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:53 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:53.563571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b4bcf-2657-41c9-854a-4a2a6903c9a8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:53.572336 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 985/1976] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:53 2022] PUT /identity/v3/projects/5a039d1f112545c6ae7ef7bbc5c972ba/users/5cf91939a71c490b81c88165df09de9e/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:53.573806 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0876d77-e056-49d7-94a3-333a7da90970 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:53.688025 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0876d77-e056-49d7-94a3-333a7da90970 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:53.714122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b4bcf-2657-41c9-854a-4a2a6903c9a8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:53.721530 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6aee0-852a-4547-b754-73177aae4416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:53.722286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6aee0-852a-4547-b754-73177aae4416 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:53.722575 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6aee0-852a-4547-b754-73177aae4416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:53.723016 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6aee0-852a-4547-b754-73177aae4416 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:53.723776 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6aee0-852a-4547-b754-73177aae4416 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:53.727478 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6aee0-852a-4547-b754-73177aae4416 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:53.774073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 992/1977] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:53 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:53.780797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6aee0-852a-4547-b754-73177aae4416 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:53.834059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6aee0-852a-4547-b754-73177aae4416 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:53.841021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae8d28-1b05-44f7-9319-4ada1513c5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:53.841722 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae8d28-1b05-44f7-9319-4ada1513c5aa admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:53.842016 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae8d28-1b05-44f7-9319-4ada1513c5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:53.842292 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae8d28-1b05-44f7-9319-4ada1513c5aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:53.843204 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae8d28-1b05-44f7-9319-4ada1513c5aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:53.847589 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae8d28-1b05-44f7-9319-4ada1513c5aa admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:53.881440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b41a8-ab3f-4b6a-9542-299a49e23510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:53.882632 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 993/1978] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:53.884507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b41a8-ab3f-4b6a-9542-299a49e23510 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:53.885443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b41a8-ab3f-4b6a-9542-299a49e23510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:53.887061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae8d28-1b05-44f7-9319-4ada1513c5aa admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:53.887549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b41a8-ab3f-4b6a-9542-299a49e23510 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:53.889041 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b41a8-ab3f-4b6a-9542-299a49e23510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:53.893322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b41a8-ab3f-4b6a-9542-299a49e23510 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:53.915869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 986/1979] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:53.921596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b41a8-ab3f-4b6a-9542-299a49e23510 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:53.930718 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae8d28-1b05-44f7-9319-4ada1513c5aa admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:53.937264 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e55f1-0671-483a-b233-1f2b6bd66ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:53.937954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e55f1-0671-483a-b233-1f2b6bd66ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:53.938241 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e55f1-0671-483a-b233-1f2b6bd66ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:53.938702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e55f1-0671-483a-b233-1f2b6bd66ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:53.939742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e55f1-0671-483a-b233-1f2b6bd66ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:53.941673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e55f1-0671-483a-b233-1f2b6bd66ef8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:53.942335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b41a8-ab3f-4b6a-9542-299a49e23510 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:53.956050 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4926a59-135c-42a0-b9fe-c4b556b779c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:53.957148 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4926a59-135c-42a0-b9fe-c4b556b779c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:53.957308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 994/1980] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:53.958212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4926a59-135c-42a0-b9fe-c4b556b779c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:53.958636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4926a59-135c-42a0-b9fe-c4b556b779c5 admin admin] PATH_INFO: `/v3/projects/5a039d1f112545c6ae7ef7bbc5c972ba/users/5cf91939a71c490b81c88165df09de9e/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:53.958828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e55f1-0671-483a-b233-1f2b6bd66ef8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:53.963332 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4926a59-135c-42a0-b9fe-c4b556b779c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a039d1f112545c6ae7ef7bbc5c972ba, user_id=5cf91939a71c490b81c88165df09de9e, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:53.966203 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4926a59-135c-42a0-b9fe-c4b556b779c5 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:53.969160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e55f1-0671-483a-b233-1f2b6bd66ef8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:53.975310 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f38a8-133d-4024-9d4b-53b77ac1d0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:53.976212 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f38a8-133d-4024-9d4b-53b77ac1d0a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:53.976677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f38a8-133d-4024-9d4b-53b77ac1d0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:53.976966 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f38a8-133d-4024-9d4b-53b77ac1d0a1 admin admin] PATH_INFO: `/v3/projects/38318668926943bab6bd1aa121ffd503/users/0c5ee88cd213478697c040ef84b4a1d2/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:53.994127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 987/1981] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:53 2022] PUT /identity/v3/projects/5a039d1f112545c6ae7ef7bbc5c972ba/users/5cf91939a71c490b81c88165df09de9e/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:54.008638 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c61172-5ced-40d0-bde4-967612b9b90d None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:54.009079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c61172-5ced-40d0-bde4-967612b9b90d None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:54.009891 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c61172-5ced-40d0-bde4-967612b9b90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:54.012039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f38a8-133d-4024-9d4b-53b77ac1d0a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38318668926943bab6bd1aa121ffd503, user_id=0c5ee88cd213478697c040ef84b4a1d2, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:54.020063 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f38a8-133d-4024-9d4b-53b77ac1d0a1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:54.049082 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 995/1982] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:53 2022] PUT /identity/v3/projects/38318668926943bab6bd1aa121ffd503/users/0c5ee88cd213478697c040ef84b4a1d2/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:54.064094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828cc09-5848-4db6-a008-0bcfd68999bc None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:54.064657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828cc09-5848-4db6-a008-0bcfd68999bc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:54.065029 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828cc09-5848-4db6-a008-0bcfd68999bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:54.076429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-99c61172-5ced-40d0-bde4-967612b9b90d None None] MFA Rules not processed for user `5cf91939a71c490b81c88165df09de9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:54.081159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-99c61172-5ced-40d0-bde4-967612b9b90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:54.110827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:54.141692 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3828cc09-5848-4db6-a008-0bcfd68999bc None None] MFA Rules not processed for user `0c5ee88cd213478697c040ef84b4a1d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:54.144517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3828cc09-5848-4db6-a008-0bcfd68999bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:54.215240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:54.260795 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 996/1983] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:54.263287 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 988/1984] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:54.267490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828cc09-5848-4db6-a008-0bcfd68999bc None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:54.276709 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c61172-5ced-40d0-bde4-967612b9b90d None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:54.286013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-99c61172-5ced-40d0-bde4-967612b9b90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:54.426594 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c61172-5ced-40d0-bde4-967612b9b90d None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:54.432460 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788790e-9730-425f-a4e7-bb76288610a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:54.433504 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788790e-9730-425f-a4e7-bb76288610a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:54.433948 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788790e-9730-425f-a4e7-bb76288610a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:54.434670 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788790e-9730-425f-a4e7-bb76288610a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:54.441422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828cc09-5848-4db6-a008-0bcfd68999bc None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:54.445670 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788790e-9730-425f-a4e7-bb76288610a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:54.447354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788790e-9730-425f-a4e7-bb76288610a1 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:54.448016 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe235095-1923-40d1-9f55-fa570827db61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:54.448871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe235095-1923-40d1-9f55-fa570827db61 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:54.449159 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe235095-1923-40d1-9f55-fa570827db61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:54.449438 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe235095-1923-40d1-9f55-fa570827db61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:54.450195 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe235095-1923-40d1-9f55-fa570827db61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:54.456412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe235095-1923-40d1-9f55-fa570827db61 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:54.472671 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 989/1985] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:54.485610 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 997/1986] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:54 2022] POST /identity/v3/projects => generated 375 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:54.489223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788790e-9730-425f-a4e7-bb76288610a1 service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:54.501037 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788790e-9730-425f-a4e7-bb76288610a1 service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:54.611105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70117ca4-2696-4280-96e2-0cdd1c8d4326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:54.612408 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70117ca4-2696-4280-96e2-0cdd1c8d4326 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:54.612754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70117ca4-2696-4280-96e2-0cdd1c8d4326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:54.613112 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70117ca4-2696-4280-96e2-0cdd1c8d4326 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:54.616346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70117ca4-2696-4280-96e2-0cdd1c8d4326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:54.634598 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70117ca4-2696-4280-96e2-0cdd1c8d4326 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:54.675803 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 990/1987] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:54 2022] POST /identity/v3/users => generated 294 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:54.684405 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe235095-1923-40d1-9f55-fa570827db61 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:54.701998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe235095-1923-40d1-9f55-fa570827db61 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:54.710614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcc242-d080-4999-b9b5-62e8cd37a327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:54.711450 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcc242-d080-4999-b9b5-62e8cd37a327 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:54.712353 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcc242-d080-4999-b9b5-62e8cd37a327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:54.712738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcc242-d080-4999-b9b5-62e8cd37a327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:54.722431 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcc242-d080-4999-b9b5-62e8cd37a327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:54.725905 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcc242-d080-4999-b9b5-62e8cd37a327 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:54.743478 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 998/1988] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:54.747722 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70117ca4-2696-4280-96e2-0cdd1c8d4326 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:54.762437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70117ca4-2696-4280-96e2-0cdd1c8d4326 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:54.770119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ff8d5-ee61-46d8-a471-84a99a54c373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:54.771158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ff8d5-ee61-46d8-a471-84a99a54c373 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:54.771635 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ff8d5-ee61-46d8-a471-84a99a54c373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:54.771850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ff8d5-ee61-46d8-a471-84a99a54c373 admin admin] PATH_INFO: `/v3/projects/52e44c059a894a28ad60514bc6ddfe15/users/0ed78e5708ff402db886b2bd62d9e4ce/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:54.792135 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ff8d5-ee61-46d8-a471-84a99a54c373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52e44c059a894a28ad60514bc6ddfe15, user_id=0ed78e5708ff402db886b2bd62d9e4ce, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:54.795161 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ff8d5-ee61-46d8-a471-84a99a54c373 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:54.828146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 991/1989] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:54 2022] PUT /identity/v3/projects/52e44c059a894a28ad60514bc6ddfe15/users/0ed78e5708ff402db886b2bd62d9e4ce/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:54.833232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcc242-d080-4999-b9b5-62e8cd37a327 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:54.846917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcc242-d080-4999-b9b5-62e8cd37a327 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:54.967676 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955f5bc-ea4a-4d6f-b9c2-caaaac2007f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:54.970840 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955f5bc-ea4a-4d6f-b9c2-caaaac2007f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:54.971319 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955f5bc-ea4a-4d6f-b9c2-caaaac2007f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:54.971746 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955f5bc-ea4a-4d6f-b9c2-caaaac2007f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:54.973643 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955f5bc-ea4a-4d6f-b9c2-caaaac2007f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:54.978306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955f5bc-ea4a-4d6f-b9c2-caaaac2007f3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:55.004443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ff8d5-ee61-46d8-a471-84a99a54c373 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:55.007151 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 999/1990] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:55.029204 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ff8d5-ee61-46d8-a471-84a99a54c373 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:55.042847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef20bb-3f10-44c5-9d1b-9e979ca422fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:55.043789 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef20bb-3f10-44c5-9d1b-9e979ca422fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:55.044079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef20bb-3f10-44c5-9d1b-9e979ca422fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:55.044380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef20bb-3f10-44c5-9d1b-9e979ca422fc admin admin] PATH_INFO: `/v3/projects/52e44c059a894a28ad60514bc6ddfe15/users/0ed78e5708ff402db886b2bd62d9e4ce/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:55.052088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef20bb-3f10-44c5-9d1b-9e979ca422fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=52e44c059a894a28ad60514bc6ddfe15, user_id=0ed78e5708ff402db886b2bd62d9e4ce, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:55.055518 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef20bb-3f10-44c5-9d1b-9e979ca422fc admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:55.079874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 992/1991] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:55 2022] PUT /identity/v3/projects/52e44c059a894a28ad60514bc6ddfe15/users/0ed78e5708ff402db886b2bd62d9e4ce/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:55.094709 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86b946-3443-4427-9561-64cb77b0096f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:55.095316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86b946-3443-4427-9561-64cb77b0096f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:55.095316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86b946-3443-4427-9561-64cb77b0096f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:55.148040 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-cc86b946-3443-4427-9561-64cb77b0096f None None] MFA Rules not processed for user `0ed78e5708ff402db886b2bd62d9e4ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:55.149516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cc86b946-3443-4427-9561-64cb77b0096f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:55.263084 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:55.363324 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1000/1992] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:55.373039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef20bb-3f10-44c5-9d1b-9e979ca422fc admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:55.379528 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:55.474123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef20bb-3f10-44c5-9d1b-9e979ca422fc admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:55.478721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85165fe9-8cc1-43fd-b7dd-86055e6bde4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:55.479432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85165fe9-8cc1-43fd-b7dd-86055e6bde4a service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:55.479865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85165fe9-8cc1-43fd-b7dd-86055e6bde4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:55.480150 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85165fe9-8cc1-43fd-b7dd-86055e6bde4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:55.590217 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85165fe9-8cc1-43fd-b7dd-86055e6bde4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:55.591681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85165fe9-8cc1-43fd-b7dd-86055e6bde4a service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:55.603778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 993/1993] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:55.712874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86b946-3443-4427-9561-64cb77b0096f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:55.822886 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86b946-3443-4427-9561-64cb77b0096f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:55.832606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062afea-1925-4d95-a82d-ff0a59ca13fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:55.834681 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062afea-1925-4d95-a82d-ff0a59ca13fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:55.834969 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062afea-1925-4d95-a82d-ff0a59ca13fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:55.836043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062afea-1925-4d95-a82d-ff0a59ca13fa admin admin] PATH_INFO: `/v3/users/cf1744f4712942a9b1244f07ed543943` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:55.837465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062afea-1925-4d95-a82d-ff0a59ca13fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf1744f4712942a9b1244f07ed543943)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:55.839486 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062afea-1925-4d95-a82d-ff0a59ca13fa admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:55.948409 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e062afea-1925-4d95-a82d-ff0a59ca13fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf1744f4712942a9b1244f07ed543943'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:55.973813 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e062afea-1925-4d95-a82d-ff0a59ca13fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf1744f4712942a9b1244f07ed543943'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:55.975318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e062afea-1925-4d95-a82d-ff0a59ca13fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf1744f4712942a9b1244f07ed543943'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:55.985920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e062afea-1925-4d95-a82d-ff0a59ca13fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf1744f4712942a9b1244f07ed543943'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:56.014196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e062afea-1925-4d95-a82d-ff0a59ca13fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf1744f4712942a9b1244f07ed543943'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:56.041220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1001/1994] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:55 2022] DELETE /identity/v3/users/cf1744f4712942a9b1244f07ed543943 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:56.045782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d51d5f-fd35-487d-8cc4-92f0f009b0dc None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:56.046872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d51d5f-fd35-487d-8cc4-92f0f009b0dc None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:56.047361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d51d5f-fd35-487d-8cc4-92f0f009b0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:56.082269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-a9d51d5f-fd35-487d-8cc4-92f0f009b0dc None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:56.088076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-a9d51d5f-fd35-487d-8cc4-92f0f009b0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:56.211130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062afea-1925-4d95-a82d-ff0a59ca13fa admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:56.212733 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e062afea-1925-4d95-a82d-ff0a59ca13fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:56.285970 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 994/1995] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:56.288795 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d51d5f-fd35-487d-8cc4-92f0f009b0dc None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:56.292555 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-a9d51d5f-fd35-487d-8cc4-92f0f009b0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:56.334678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:56.348103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062afea-1925-4d95-a82d-ff0a59ca13fa admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:56.369498 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d5716-689a-4c52-bbef-05dbdc640ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:56.370410 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d5716-689a-4c52-bbef-05dbdc640ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:56.370694 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d5716-689a-4c52-bbef-05dbdc640ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:56.370970 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d5716-689a-4c52-bbef-05dbdc640ffe admin admin] PATH_INFO: `/v3/users/59a8d6ff130c4b7b9ee5460fa56dcca5` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:56.372252 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d5716-689a-4c52-bbef-05dbdc640ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=59a8d6ff130c4b7b9ee5460fa56dcca5)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:56.374265 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d5716-689a-4c52-bbef-05dbdc640ffe admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:56.422127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d51d5f-fd35-487d-8cc4-92f0f009b0dc None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:56.427153 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d1bef-0a95-4967-962e-ad62399ff65f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:56.427975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d1bef-0a95-4967-962e-ad62399ff65f service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:56.428362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d1bef-0a95-4967-962e-ad62399ff65f service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:56.428701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d1bef-0a95-4967-962e-ad62399ff65f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:56.441968 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d1bef-0a95-4967-962e-ad62399ff65f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:56.443966 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d1bef-0a95-4967-962e-ad62399ff65f service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:56.459426 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d05d5716-689a-4c52-bbef-05dbdc640ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59a8d6ff130c4b7b9ee5460fa56dcca5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:56.464743 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 995/1996] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:56.470788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d05d5716-689a-4c52-bbef-05dbdc640ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59a8d6ff130c4b7b9ee5460fa56dcca5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:56.471570 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d05d5716-689a-4c52-bbef-05dbdc640ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59a8d6ff130c4b7b9ee5460fa56dcca5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:56.481867 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d05d5716-689a-4c52-bbef-05dbdc640ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59a8d6ff130c4b7b9ee5460fa56dcca5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:56.504337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d05d5716-689a-4c52-bbef-05dbdc640ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59a8d6ff130c4b7b9ee5460fa56dcca5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:56.515328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1002/1997] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:56 2022] DELETE /identity/v3/users/59a8d6ff130c4b7b9ee5460fa56dcca5 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:56.517323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3746282-fe9d-4228-b121-1b5b5750fc83 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:56.517623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3746282-fe9d-4228-b121-1b5b5750fc83 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:56.517907 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3746282-fe9d-4228-b121-1b5b5750fc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:56.524436 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:56.542607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-d3746282-fe9d-4228-b121-1b5b5750fc83 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:56.547046 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d3746282-fe9d-4228-b121-1b5b5750fc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:56.621487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 996/1998] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:56.635093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d5716-689a-4c52-bbef-05dbdc640ffe admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:56.636837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d05d5716-689a-4c52-bbef-05dbdc640ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:56.704365 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d5716-689a-4c52-bbef-05dbdc640ffe admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:56.709055 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa82107-60f4-4ea2-9427-f2b18dbf5452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:56.709798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa82107-60f4-4ea2-9427-f2b18dbf5452 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:56.710089 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa82107-60f4-4ea2-9427-f2b18dbf5452 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:56.710368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa82107-60f4-4ea2-9427-f2b18dbf5452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:56.726053 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa82107-60f4-4ea2-9427-f2b18dbf5452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:56.727978 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa82107-60f4-4ea2-9427-f2b18dbf5452 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:56.739116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1003/1999] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:56.875063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3746282-fe9d-4228-b121-1b5b5750fc83 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:56.876901 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d3746282-fe9d-4228-b121-1b5b5750fc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:56.891262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3746282-fe9d-4228-b121-1b5b5750fc83 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:56.898427 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fabce7-a524-496f-ac44-260254677903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:56.899221 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fabce7-a524-496f-ac44-260254677903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:56.899499 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fabce7-a524-496f-ac44-260254677903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:56.899782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fabce7-a524-496f-ac44-260254677903 admin admin] PATH_INFO: `/v3/projects/05a141a07ae04d079500a197ec5a8111` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:56.901173 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fabce7-a524-496f-ac44-260254677903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05a141a07ae04d079500a197ec5a8111)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:56.903415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fabce7-a524-496f-ac44-260254677903 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:56.986499 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-92fabce7-a524-496f-ac44-260254677903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05a141a07ae04d079500a197ec5a8111', 'request_id': 'req-92fabce7-a524-496f-ac44-260254677903'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:56.997384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-92fabce7-a524-496f-ac44-260254677903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05a141a07ae04d079500a197ec5a8111', 'request_id': 'req-92fabce7-a524-496f-ac44-260254677903'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.006131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-92fabce7-a524-496f-ac44-260254677903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05a141a07ae04d079500a197ec5a8111', 'request_id': 'req-92fabce7-a524-496f-ac44-260254677903'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.018885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-92fabce7-a524-496f-ac44-260254677903 admin admin] The token cache is being invalidate because project 05a141a07ae04d079500a197ec5a8111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:57.019246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-92fabce7-a524-496f-ac44-260254677903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.020095 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-92fabce7-a524-496f-ac44-260254677903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.022464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 997/2000] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:56 2022] DELETE /identity/v3/projects/05a141a07ae04d079500a197ec5a8111 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:57.026710 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa82107-60f4-4ea2-9427-f2b18dbf5452 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:57.038858 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa82107-60f4-4ea2-9427-f2b18dbf5452 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:57.103033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5251c-a5ac-45d0-8001-f0f7808f574d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:57.103915 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5251c-a5ac-45d0-8001-f0f7808f574d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:57.104200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5251c-a5ac-45d0-8001-f0f7808f574d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:57.104553 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5251c-a5ac-45d0-8001-f0f7808f574d admin admin] PATH_INFO: `/v3/users/5cf91939a71c490b81c88165df09de9e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:57.106325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5251c-a5ac-45d0-8001-f0f7808f574d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cf91939a71c490b81c88165df09de9e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:57.107993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5251c-a5ac-45d0-8001-f0f7808f574d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:57.125815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fabce7-a524-496f-ac44-260254677903 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:57.127977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-92fabce7-a524-496f-ac44-260254677903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:57.155865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fabce7-a524-496f-ac44-260254677903 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:57.162148 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64456d-3218-4928-80c7-884899b76df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:57.163017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64456d-3218-4928-80c7-884899b76df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:57.167525 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64456d-3218-4928-80c7-884899b76df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:57.167678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:57.167759 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64456d-3218-4928-80c7-884899b76df0 admin admin] PATH_INFO: `/v3/projects/dfdff1e88a544e60993c77327684f672` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:57.169128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64456d-3218-4928-80c7-884899b76df0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfdff1e88a544e60993c77327684f672)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:57.172429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64456d-3218-4928-80c7-884899b76df0 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:57.215464 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dff5251c-a5ac-45d0-8001-f0f7808f574d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cf91939a71c490b81c88165df09de9e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.234259 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dff5251c-a5ac-45d0-8001-f0f7808f574d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cf91939a71c490b81c88165df09de9e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.235069 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dff5251c-a5ac-45d0-8001-f0f7808f574d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cf91939a71c490b81c88165df09de9e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.244702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dff5251c-a5ac-45d0-8001-f0f7808f574d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cf91939a71c490b81c88165df09de9e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.267868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dff5251c-a5ac-45d0-8001-f0f7808f574d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cf91939a71c490b81c88165df09de9e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.270009 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-be64456d-3218-4928-80c7-884899b76df0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfdff1e88a544e60993c77327684f672', 'request_id': 'req-be64456d-3218-4928-80c7-884899b76df0'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.277579 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1004/2001] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:57 2022] DELETE /identity/v3/users/5cf91939a71c490b81c88165df09de9e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:57.282097 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-be64456d-3218-4928-80c7-884899b76df0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfdff1e88a544e60993c77327684f672', 'request_id': 'req-be64456d-3218-4928-80c7-884899b76df0'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.290787 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-be64456d-3218-4928-80c7-884899b76df0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfdff1e88a544e60993c77327684f672', 'request_id': 'req-be64456d-3218-4928-80c7-884899b76df0'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.321323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-be64456d-3218-4928-80c7-884899b76df0 admin admin] The token cache is being invalidate because project dfdff1e88a544e60993c77327684f672 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:57.321619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-be64456d-3218-4928-80c7-884899b76df0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.322632 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-be64456d-3218-4928-80c7-884899b76df0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.328781 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 998/2002] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:57 2022] DELETE /identity/v3/projects/dfdff1e88a544e60993c77327684f672 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:57.330546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8dcd3-db34-47ad-b1d4-a902f05ab124 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:57.330915 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8dcd3-db34-47ad-b1d4-a902f05ab124 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:57.331205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8dcd3-db34-47ad-b1d4-a902f05ab124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:57.351028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-58c8dcd3-db34-47ad-b1d4-a902f05ab124 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:57.352012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-58c8dcd3-db34-47ad-b1d4-a902f05ab124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:57.465038 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1005/2003] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:57.468176 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d7a9a-781c-4e95-bb52-7bfc580feb41 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:57.468701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d7a9a-781c-4e95-bb52-7bfc580feb41 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:57.469321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d7a9a-781c-4e95-bb52-7bfc580feb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:57.484170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8dcd3-db34-47ad-b1d4-a902f05ab124 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:57.485608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-58c8dcd3-db34-47ad-b1d4-a902f05ab124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:57.490408 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-ce5d7a9a-781c-4e95-bb52-7bfc580feb41 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:57.491665 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ce5d7a9a-781c-4e95-bb52-7bfc580feb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:57.503908 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 999/2004] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:57.505122 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d7a9a-781c-4e95-bb52-7bfc580feb41 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:57.510221 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:57.517972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d7a9a-781c-4e95-bb52-7bfc580feb41 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:57.530019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629ae32-ba12-4c71-868d-e150d5290bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:57.530713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629ae32-ba12-4c71-868d-e150d5290bac admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:57.531029 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629ae32-ba12-4c71-868d-e150d5290bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:57.531277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629ae32-ba12-4c71-868d-e150d5290bac admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:57.531997 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629ae32-ba12-4c71-868d-e150d5290bac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:57.532365 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629ae32-ba12-4c71-868d-e150d5290bac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:57.533884 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629ae32-ba12-4c71-868d-e150d5290bac admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:57.549774 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1000/2005] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:57.551443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629ae32-ba12-4c71-868d-e150d5290bac admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:57.564413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629ae32-ba12-4c71-868d-e150d5290bac admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:57.582872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f29381-e627-4cb1-bec5-9e6e0040c660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:57.584159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f29381-e627-4cb1-bec5-9e6e0040c660 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:57.584498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f29381-e627-4cb1-bec5-9e6e0040c660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:57.584778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f29381-e627-4cb1-bec5-9e6e0040c660 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:57.585542 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f29381-e627-4cb1-bec5-9e6e0040c660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:57.587548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f29381-e627-4cb1-bec5-9e6e0040c660 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:57.593391 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8dcd3-db34-47ad-b1d4-a902f05ab124 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:57.597788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d30773-1171-44bc-a726-57987c9c4d0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:57.598618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d30773-1171-44bc-a726-57987c9c4d0c service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:57.598979 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d30773-1171-44bc-a726-57987c9c4d0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:57.600060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d30773-1171-44bc-a726-57987c9c4d0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:57.602688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-27d30773-1171-44bc-a726-57987c9c4d0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:57.622827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1001/2006] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:57 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:57.623858 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f29381-e627-4cb1-bec5-9e6e0040c660 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:57.672793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:57.691492 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d30773-1171-44bc-a726-57987c9c4d0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:57.694013 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d30773-1171-44bc-a726-57987c9c4d0c service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:57.699075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f29381-e627-4cb1-bec5-9e6e0040c660 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:57.704281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25730e-a88c-4cad-9b73-bd3c55139303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:57.705171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25730e-a88c-4cad-9b73-bd3c55139303 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:57.705496 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25730e-a88c-4cad-9b73-bd3c55139303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:57.705672 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1006/2007] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:57.706975 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d30773-1171-44bc-a726-57987c9c4d0c service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:57.707552 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25730e-a88c-4cad-9b73-bd3c55139303 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:57.708371 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25730e-a88c-4cad-9b73-bd3c55139303 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:57.709595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-27d30773-1171-44bc-a726-57987c9c4d0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:57.710906 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25730e-a88c-4cad-9b73-bd3c55139303 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:57.725586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d30773-1171-44bc-a726-57987c9c4d0c service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:57.730856 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2b203-eb1e-445a-971c-ae25e4a0ef47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:57.731728 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2b203-eb1e-445a-971c-ae25e4a0ef47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:57.732010 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2b203-eb1e-445a-971c-ae25e4a0ef47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:57.732281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2b203-eb1e-445a-971c-ae25e4a0ef47 admin admin] PATH_INFO: `/v3/projects/5a039d1f112545c6ae7ef7bbc5c972ba` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:57.733656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1002/2008] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:57 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:57.733776 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2b203-eb1e-445a-971c-ae25e4a0ef47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a039d1f112545c6ae7ef7bbc5c972ba)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:57.739036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25730e-a88c-4cad-9b73-bd3c55139303 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:57.739596 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2b203-eb1e-445a-971c-ae25e4a0ef47 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:57.749958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25730e-a88c-4cad-9b73-bd3c55139303 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:57.755369 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037e02f-d5b2-4daf-b974-40b0204de21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:57.756749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037e02f-d5b2-4daf-b974-40b0204de21a admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:57.757041 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037e02f-d5b2-4daf-b974-40b0204de21a admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:57.758034 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037e02f-d5b2-4daf-b974-40b0204de21a admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:57.759170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037e02f-d5b2-4daf-b974-40b0204de21a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:57.760978 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037e02f-d5b2-4daf-b974-40b0204de21a admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:57.772404 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1003/2009] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:57.776245 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037e02f-d5b2-4daf-b974-40b0204de21a admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:57.793358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037e02f-d5b2-4daf-b974-40b0204de21a admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:57.805769 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b1e2b203-eb1e-445a-971c-ae25e4a0ef47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a039d1f112545c6ae7ef7bbc5c972ba', 'request_id': 'req-b1e2b203-eb1e-445a-971c-ae25e4a0ef47'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.812862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b1e2b203-eb1e-445a-971c-ae25e4a0ef47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a039d1f112545c6ae7ef7bbc5c972ba', 'request_id': 'req-b1e2b203-eb1e-445a-971c-ae25e4a0ef47'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.823841 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b1e2b203-eb1e-445a-971c-ae25e4a0ef47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a039d1f112545c6ae7ef7bbc5c972ba', 'request_id': 'req-b1e2b203-eb1e-445a-971c-ae25e4a0ef47'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.837630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b1e2b203-eb1e-445a-971c-ae25e4a0ef47 admin admin] The token cache is being invalidate because project 5a039d1f112545c6ae7ef7bbc5c972ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:57.838019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b1e2b203-eb1e-445a-971c-ae25e4a0ef47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.838691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b1e2b203-eb1e-445a-971c-ae25e4a0ef47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:57.840256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1007/2010] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:57 2022] DELETE /identity/v3/projects/5a039d1f112545c6ae7ef7bbc5c972ba => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:57.845069 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7627fb7-617c-4092-bbf9-bba38403a1a7 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:57.845421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7627fb7-617c-4092-bbf9-bba38403a1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:57.845766 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7627fb7-617c-4092-bbf9-bba38403a1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:57.864572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c7627fb7-617c-4092-bbf9-bba38403a1a7 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:57.865806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c7627fb7-617c-4092-bbf9-bba38403a1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:57.871517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee1b97-7b74-4b8b-8947-86a3fab44636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:57.872323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee1b97-7b74-4b8b-8947-86a3fab44636 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:57.872531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee1b97-7b74-4b8b-8947-86a3fab44636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:57.872825 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee1b97-7b74-4b8b-8947-86a3fab44636 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:57.874235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee1b97-7b74-4b8b-8947-86a3fab44636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:57.875339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1008/2011] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:57.876458 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee1b97-7b74-4b8b-8947-86a3fab44636 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:57.883920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6af85-4666-41c0-be08-c12d8432b9fd None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:57.884457 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6af85-4666-41c0-be08-c12d8432b9fd None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:57.884905 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6af85-4666-41c0-be08-c12d8432b9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:57.896680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1004/2012] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:57.899487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee1b97-7b74-4b8b-8947-86a3fab44636 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:57.901605 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-fcee1b97-7b74-4b8b-8947-86a3fab44636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:57.904246 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0ec6af85-4666-41c0-be08-c12d8432b9fd None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:57.905449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0ec6af85-4666-41c0-be08-c12d8432b9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:57.916983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1009/2013] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:57.917128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee1b97-7b74-4b8b-8947-86a3fab44636 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:57.917970 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6af85-4666-41c0-be08-c12d8432b9fd None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:57.919819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e875d5f-bd2f-49db-ad11-aa77be7ef20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:57.920646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e875d5f-bd2f-49db-ad11-aa77be7ef20a admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:57.920932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e875d5f-bd2f-49db-ad11-aa77be7ef20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:57.921208 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e875d5f-bd2f-49db-ad11-aa77be7ef20a admin admin] PATH_INFO: `/v3/projects/182d5b6ac9e6412b907119a623869c80/users/4a4f4d585adf40b880be44c29ecd53cd/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:57.929405 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6af85-4666-41c0-be08-c12d8432b9fd None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:57.935726 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6572be-4123-438c-bbda-f245584afbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:57.935859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6572be-4123-438c-bbda-f245584afbba admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:57.936078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6572be-4123-438c-bbda-f245584afbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:57.936531 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6572be-4123-438c-bbda-f245584afbba admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:57.937563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6572be-4123-438c-bbda-f245584afbba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:57.937821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6572be-4123-438c-bbda-f245584afbba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:57.938508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e875d5f-bd2f-49db-ad11-aa77be7ef20a admin admin] RBAC: Authorizing `identity:create_grant(project_id=182d5b6ac9e6412b907119a623869c80, user_id=4a4f4d585adf40b880be44c29ecd53cd, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:57.939800 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6572be-4123-438c-bbda-f245584afbba admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:57.941639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e875d5f-bd2f-49db-ad11-aa77be7ef20a admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:57.956961 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1010/2014] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:24:57.957548 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1005/2015] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:57 2022] PUT /identity/v3/projects/182d5b6ac9e6412b907119a623869c80/users/4a4f4d585adf40b880be44c29ecd53cd/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:57.963427 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec004e6-6893-4b87-b097-848f2f4bca1e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:57.963714 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec004e6-6893-4b87-b097-848f2f4bca1e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:57.963989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec004e6-6893-4b87-b097-848f2f4bca1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:57.965213 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e875d5f-bd2f-49db-ad11-aa77be7ef20a admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:57.986549 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4ec004e6-6893-4b87-b097-848f2f4bca1e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:57.987793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4ec004e6-6893-4b87-b097-848f2f4bca1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:57.990880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:58.057061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e875d5f-bd2f-49db-ad11-aa77be7ef20a admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:58.058894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1011/2016] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:58.062481 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fead287-c3c9-4714-a6fe-7c78945c7052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:58.063164 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fead287-c3c9-4714-a6fe-7c78945c7052 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:58.063453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fead287-c3c9-4714-a6fe-7c78945c7052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:58.063733 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fead287-c3c9-4714-a6fe-7c78945c7052 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:58.064473 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fead287-c3c9-4714-a6fe-7c78945c7052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:58.066380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fead287-c3c9-4714-a6fe-7c78945c7052 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:58.069508 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec004e6-6893-4b87-b097-848f2f4bca1e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:58.071237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4ec004e6-6893-4b87-b097-848f2f4bca1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:58.097587 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1006/2017] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:57 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:58.098656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fead287-c3c9-4714-a6fe-7c78945c7052 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:58.138349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:58.142615 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec004e6-6893-4b87-b097-848f2f4bca1e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:58.162324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fead287-c3c9-4714-a6fe-7c78945c7052 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:58.167258 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95787edf-6368-4450-a170-f4f7e5dc03d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:58.167873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95787edf-6368-4450-a170-f4f7e5dc03d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:58.168164 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95787edf-6368-4450-a170-f4f7e5dc03d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:58.168579 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95787edf-6368-4450-a170-f4f7e5dc03d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:58.169288 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95787edf-6368-4450-a170-f4f7e5dc03d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:58.171239 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95787edf-6368-4450-a170-f4f7e5dc03d9 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:58.192310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1007/2018] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:58 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:58.197164 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e2f97-6386-4942-a860-95cdc22b865c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:58.197352 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95787edf-6368-4450-a170-f4f7e5dc03d9 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:58.197825 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e2f97-6386-4942-a860-95cdc22b865c service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:58.198116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e2f97-6386-4942-a860-95cdc22b865c service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:58.198408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e2f97-6386-4942-a860-95cdc22b865c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:58.206867 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95787edf-6368-4450-a170-f4f7e5dc03d9 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:58.208583 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e2f97-6386-4942-a860-95cdc22b865c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:58.210021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e2f97-6386-4942-a860-95cdc22b865c service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:58.211599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e28c99-5108-4fef-9b81-98eaf1d2cbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:58.212265 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e28c99-5108-4fef-9b81-98eaf1d2cbfd admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:58.212585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e28c99-5108-4fef-9b81-98eaf1d2cbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:58.212879 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e28c99-5108-4fef-9b81-98eaf1d2cbfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:58.213622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e28c99-5108-4fef-9b81-98eaf1d2cbfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:58.215209 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e28c99-5108-4fef-9b81-98eaf1d2cbfd admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:58.220243 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1012/2019] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:58.225744 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1008/2020] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:58.229902 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e2f97-6386-4942-a860-95cdc22b865c service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:58.239263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e2f97-6386-4942-a860-95cdc22b865c service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:58.246724 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69068f9-d63e-490f-8f31-790fdf308f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:58.247391 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69068f9-d63e-490f-8f31-790fdf308f3e admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:58.247674 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69068f9-d63e-490f-8f31-790fdf308f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:58.247947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69068f9-d63e-490f-8f31-790fdf308f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:58.248787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69068f9-d63e-490f-8f31-790fdf308f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:58.250542 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69068f9-d63e-490f-8f31-790fdf308f3e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:58.270574 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1013/2021] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:58.274847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e28c99-5108-4fef-9b81-98eaf1d2cbfd admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:58.293714 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e28c99-5108-4fef-9b81-98eaf1d2cbfd admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:58.299563 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba25901-e65b-45a3-bd7f-6259c15e0e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:58.300874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba25901-e65b-45a3-bd7f-6259c15e0e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:58.301457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba25901-e65b-45a3-bd7f-6259c15e0e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:58.301457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba25901-e65b-45a3-bd7f-6259c15e0e4e admin admin] PATH_INFO: `/v3/projects/99fed5923d4e4c8a8494e40ef86e7a8c/users/b74aaa86c6744bd5b9ed139c8a0a5b26/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:58.320394 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba25901-e65b-45a3-bd7f-6259c15e0e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=99fed5923d4e4c8a8494e40ef86e7a8c, user_id=b74aaa86c6744bd5b9ed139c8a0a5b26, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:58.323095 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba25901-e65b-45a3-bd7f-6259c15e0e4e admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:58.348591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1009/2022] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:58 2022] PUT /identity/v3/projects/99fed5923d4e4c8a8494e40ef86e7a8c/users/b74aaa86c6744bd5b9ed139c8a0a5b26/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:58.360885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c879821-88d2-4ac7-8424-84567c9cb436 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:58.361263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c879821-88d2-4ac7-8424-84567c9cb436 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:58.361661 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c879821-88d2-4ac7-8424-84567c9cb436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:58.392860 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8c879821-88d2-4ac7-8424-84567c9cb436 None None] MFA Rules not processed for user `b74aaa86c6744bd5b9ed139c8a0a5b26`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:58.394298 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8c879821-88d2-4ac7-8424-84567c9cb436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:58.485001 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1014/2023] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:58.528253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba25901-e65b-45a3-bd7f-6259c15e0e4e admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:58.537280 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5ba25901-e65b-45a3-bd7f-6259c15e0e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:58.603794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c879821-88d2-4ac7-8424-84567c9cb436 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:58.605569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8c879821-88d2-4ac7-8424-84567c9cb436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:58.623892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba25901-e65b-45a3-bd7f-6259c15e0e4e admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:58.628108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca02a1e-0024-496c-b007-1e7b5ead484b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:58.629012 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca02a1e-0024-496c-b007-1e7b5ead484b service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:58.629909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca02a1e-0024-496c-b007-1e7b5ead484b service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:58.630301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca02a1e-0024-496c-b007-1e7b5ead484b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:58.643763 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca02a1e-0024-496c-b007-1e7b5ead484b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:58.646172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca02a1e-0024-496c-b007-1e7b5ead484b service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:58.660424 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:58.675824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1010/2024] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:58.695341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca02a1e-0024-496c-b007-1e7b5ead484b service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:58.725023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c879821-88d2-4ac7-8424-84567c9cb436 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:58.733061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f26c7a-f791-4f0f-8be1-f011c9891753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:58.734116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f26c7a-f791-4f0f-8be1-f011c9891753 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:58.735063 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f26c7a-f791-4f0f-8be1-f011c9891753 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:58.735688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f26c7a-f791-4f0f-8be1-f011c9891753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:58.738423 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-28f26c7a-f791-4f0f-8be1-f011c9891753 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:58.776559 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca02a1e-0024-496c-b007-1e7b5ead484b service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:58.781469 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39be7a0-2dbe-449d-bb59-d1489c465a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:58.782338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39be7a0-2dbe-449d-bb59-d1489c465a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:58.782748 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39be7a0-2dbe-449d-bb59-d1489c465a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:58.783033 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39be7a0-2dbe-449d-bb59-d1489c465a4d admin admin] PATH_INFO: `/v3/users/b74aaa86c6744bd5b9ed139c8a0a5b26` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:58.784258 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39be7a0-2dbe-449d-bb59-d1489c465a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b74aaa86c6744bd5b9ed139c8a0a5b26)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:58.786237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39be7a0-2dbe-449d-bb59-d1489c465a4d admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:58.789998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:58.814821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f26c7a-f791-4f0f-8be1-f011c9891753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:58.817087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f26c7a-f791-4f0f-8be1-f011c9891753 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:58.829803 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1015/2025] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:58 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:58.845690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-d39be7a0-2dbe-449d-bb59-d1489c465a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b74aaa86c6744bd5b9ed139c8a0a5b26'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:58.854885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-d39be7a0-2dbe-449d-bb59-d1489c465a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b74aaa86c6744bd5b9ed139c8a0a5b26'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:58.872742 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-d39be7a0-2dbe-449d-bb59-d1489c465a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b74aaa86c6744bd5b9ed139c8a0a5b26'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:58.882008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-d39be7a0-2dbe-449d-bb59-d1489c465a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b74aaa86c6744bd5b9ed139c8a0a5b26'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:58.882828 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-d39be7a0-2dbe-449d-bb59-d1489c465a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b74aaa86c6744bd5b9ed139c8a0a5b26'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:58.891973 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1011/2026] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:58 2022] DELETE /identity/v3/users/b74aaa86c6744bd5b9ed139c8a0a5b26 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:58.895512 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850dff3-c6c6-4282-b160-f198c0597293 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:58.895812 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850dff3-c6c6-4282-b160-f198c0597293 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:58.896648 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850dff3-c6c6-4282-b160-f198c0597293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:58.915620 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e850dff3-c6c6-4282-b160-f198c0597293 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:58.916947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e850dff3-c6c6-4282-b160-f198c0597293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:58.991745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1016/2027] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:59.001903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39be7a0-2dbe-449d-bb59-d1489c465a4d admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:59.003566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d39be7a0-2dbe-449d-bb59-d1489c465a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:59.092799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39be7a0-2dbe-449d-bb59-d1489c465a4d admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:59.096796 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628f23d-7ec1-437a-ab78-ef53495fef4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:59.097476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628f23d-7ec1-437a-ab78-ef53495fef4a service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:59.097768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628f23d-7ec1-437a-ab78-ef53495fef4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:59.098052 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628f23d-7ec1-437a-ab78-ef53495fef4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:59.110612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628f23d-7ec1-437a-ab78-ef53495fef4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:59.112194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628f23d-7ec1-437a-ab78-ef53495fef4a service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:59.132774 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1012/2028] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:59.726160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850dff3-c6c6-4282-b160-f198c0597293 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:59.728836 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e850dff3-c6c6-4282-b160-f198c0597293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:59.743631 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850dff3-c6c6-4282-b160-f198c0597293 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:59.756745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b903005-b7b8-4789-94a2-b64c1e6f8256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:59.759524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b903005-b7b8-4789-94a2-b64c1e6f8256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:59.760608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b903005-b7b8-4789-94a2-b64c1e6f8256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:59.761073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b903005-b7b8-4789-94a2-b64c1e6f8256 admin admin] PATH_INFO: `/v3/projects/99fed5923d4e4c8a8494e40ef86e7a8c` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:59.763533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b903005-b7b8-4789-94a2-b64c1e6f8256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99fed5923d4e4c8a8494e40ef86e7a8c)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:59.766078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b903005-b7b8-4789-94a2-b64c1e6f8256 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:59.845237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6b903005-b7b8-4789-94a2-b64c1e6f8256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99fed5923d4e4c8a8494e40ef86e7a8c', 'request_id': 'req-6b903005-b7b8-4789-94a2-b64c1e6f8256'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:59.853771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6b903005-b7b8-4789-94a2-b64c1e6f8256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99fed5923d4e4c8a8494e40ef86e7a8c', 'request_id': 'req-6b903005-b7b8-4789-94a2-b64c1e6f8256'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:59.862331 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6b903005-b7b8-4789-94a2-b64c1e6f8256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99fed5923d4e4c8a8494e40ef86e7a8c', 'request_id': 'req-6b903005-b7b8-4789-94a2-b64c1e6f8256'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:59.877192 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6b903005-b7b8-4789-94a2-b64c1e6f8256 admin admin] The token cache is being invalidate because project 99fed5923d4e4c8a8494e40ef86e7a8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:59.877487 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6b903005-b7b8-4789-94a2-b64c1e6f8256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:59.878161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6b903005-b7b8-4789-94a2-b64c1e6f8256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:59.883687 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1017/2029] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:59 2022] DELETE /identity/v3/projects/99fed5923d4e4c8a8494e40ef86e7a8c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:59.885466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b34fb-a191-4398-9bae-5ec7906fc0ec None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:59.885925 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b34fb-a191-4398-9bae-5ec7906fc0ec None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:59.886029 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b34fb-a191-4398-9bae-5ec7906fc0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:59.911109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-e85b34fb-a191-4398-9bae-5ec7906fc0ec None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:59.913020 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e85b34fb-a191-4398-9bae-5ec7906fc0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:59.938948 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:00.004450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1013/2030] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:24:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:00.011417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cae33-870d-44c9-8c98-45d51e94d8e0 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:00.011848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cae33-870d-44c9-8c98-45d51e94d8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:00.012249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cae33-870d-44c9-8c98-45d51e94d8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:00.038313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-400cae33-870d-44c9-8c98-45d51e94d8e0 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:00.039526 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-400cae33-870d-44c9-8c98-45d51e94d8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:00.079754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1018/2031] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:00.081472 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b34fb-a191-4398-9bae-5ec7906fc0ec None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:00.105438 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b34fb-a191-4398-9bae-5ec7906fc0ec None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:00.111756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4ac77-01da-4d86-8038-db8098c8d671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:00.117247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4ac77-01da-4d86-8038-db8098c8d671 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:00.118641 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4ac77-01da-4d86-8038-db8098c8d671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:00.119064 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4ac77-01da-4d86-8038-db8098c8d671 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:00.119952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4ac77-01da-4d86-8038-db8098c8d671 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:00.120361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4ac77-01da-4d86-8038-db8098c8d671 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:00.123672 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4ac77-01da-4d86-8038-db8098c8d671 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:00.148341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1014/2032] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:25:00.152946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cae33-870d-44c9-8c98-45d51e94d8e0 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:00.166455 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cae33-870d-44c9-8c98-45d51e94d8e0 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:00.179181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d73238-b45c-4a58-8820-66a68c9038a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:00.179940 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d73238-b45c-4a58-8820-66a68c9038a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:00.180213 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d73238-b45c-4a58-8820-66a68c9038a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:00.180600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d73238-b45c-4a58-8820-66a68c9038a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:00.181564 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d73238-b45c-4a58-8820-66a68c9038a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:00.183716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d73238-b45c-4a58-8820-66a68c9038a8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:00.231376 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1019/2033] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:00 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:00.235963 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4ac77-01da-4d86-8038-db8098c8d671 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:00.251337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4ac77-01da-4d86-8038-db8098c8d671 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:00.389904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be0dce-9513-4471-a58b-3a65aa0e1ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:00.391655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be0dce-9513-4471-a58b-3a65aa0e1ccc admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:00.391958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be0dce-9513-4471-a58b-3a65aa0e1ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:00.392244 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be0dce-9513-4471-a58b-3a65aa0e1ccc admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:00.394004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be0dce-9513-4471-a58b-3a65aa0e1ccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:00.398009 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be0dce-9513-4471-a58b-3a65aa0e1ccc admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:00.457977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1015/2034] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:00 2022] POST /identity/v3/users => generated 287 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:00.462496 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d73238-b45c-4a58-8820-66a68c9038a8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:00.482318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d73238-b45c-4a58-8820-66a68c9038a8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:00.489297 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f100d-f96e-4324-9542-06d4c416ff8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:00.490657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f100d-f96e-4324-9542-06d4c416ff8e admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:00.491041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f100d-f96e-4324-9542-06d4c416ff8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:00.491861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f100d-f96e-4324-9542-06d4c416ff8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:00.493110 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f100d-f96e-4324-9542-06d4c416ff8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:00.494995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f100d-f96e-4324-9542-06d4c416ff8e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:00.513790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1020/2035] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:00.515514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be0dce-9513-4471-a58b-3a65aa0e1ccc admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:00.529347 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be0dce-9513-4471-a58b-3a65aa0e1ccc admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:00.539073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa81d5c-74d7-4fe5-b821-fd1ae0ce2bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:00.539918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa81d5c-74d7-4fe5-b821-fd1ae0ce2bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:00.540279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa81d5c-74d7-4fe5-b821-fd1ae0ce2bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:00.541490 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa81d5c-74d7-4fe5-b821-fd1ae0ce2bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:00.542320 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa81d5c-74d7-4fe5-b821-fd1ae0ce2bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:00.544072 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa81d5c-74d7-4fe5-b821-fd1ae0ce2bd8 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:00.565094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1016/2036] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:00.566930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f100d-f96e-4324-9542-06d4c416ff8e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:00.586729 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f100d-f96e-4324-9542-06d4c416ff8e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:00.591971 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd55fe-8166-4672-8976-4efcf6c3db47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:00.593146 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd55fe-8166-4672-8976-4efcf6c3db47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:00.593737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd55fe-8166-4672-8976-4efcf6c3db47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:00.594180 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd55fe-8166-4672-8976-4efcf6c3db47 admin admin] PATH_INFO: `/v3/projects/4c12e93a21a8436db1bb7276b1d3b8f6/users/bb8c4ec33399483392d80fb82a6d50eb/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:00.618235 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd55fe-8166-4672-8976-4efcf6c3db47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c12e93a21a8436db1bb7276b1d3b8f6, user_id=bb8c4ec33399483392d80fb82a6d50eb, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:00.621298 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd55fe-8166-4672-8976-4efcf6c3db47 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:00.624814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:00.660524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1021/2037] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:00 2022] PUT /identity/v3/projects/4c12e93a21a8436db1bb7276b1d3b8f6/users/bb8c4ec33399483392d80fb82a6d50eb/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:00.661236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77072cdc-09c2-4f6d-8b7c-85496939dd29 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:00.661535 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77072cdc-09c2-4f6d-8b7c-85496939dd29 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:00.661855 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77072cdc-09c2-4f6d-8b7c-85496939dd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:00.694265 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-77072cdc-09c2-4f6d-8b7c-85496939dd29 None None] MFA Rules not processed for user `bb8c4ec33399483392d80fb82a6d50eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:00.695734 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-77072cdc-09c2-4f6d-8b7c-85496939dd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:00.779122 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1017/2038] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:00.783511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd55fe-8166-4672-8976-4efcf6c3db47 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:00.899449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd55fe-8166-4672-8976-4efcf6c3db47 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:00.929248 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14706e90-069f-41f7-bc80-421c55a9e304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:00.933124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14706e90-069f-41f7-bc80-421c55a9e304 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:00.933506 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14706e90-069f-41f7-bc80-421c55a9e304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:00.934569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14706e90-069f-41f7-bc80-421c55a9e304 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:00.939086 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14706e90-069f-41f7-bc80-421c55a9e304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:00.942025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14706e90-069f-41f7-bc80-421c55a9e304 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:01.000786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1022/2039] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:00 2022] POST /identity/v3/projects => generated 375 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:01.004131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77072cdc-09c2-4f6d-8b7c-85496939dd29 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:01.021856 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77072cdc-09c2-4f6d-8b7c-85496939dd29 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:01.039681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:01.100553 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c4590-460d-4747-9f89-c8fb8634831c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:01.101485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c4590-460d-4747-9f89-c8fb8634831c admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:01.101610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c4590-460d-4747-9f89-c8fb8634831c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:01.101796 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c4590-460d-4747-9f89-c8fb8634831c admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:01.102517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c4590-460d-4747-9f89-c8fb8634831c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:01.104504 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c4590-460d-4747-9f89-c8fb8634831c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:01.158423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1018/2040] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:01 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:01.160435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14706e90-069f-41f7-bc80-421c55a9e304 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:01.173936 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14706e90-069f-41f7-bc80-421c55a9e304 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:01.181131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee841b3-81c5-446f-884d-052f303cd8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:01.182232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee841b3-81c5-446f-884d-052f303cd8ba admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:01.182533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee841b3-81c5-446f-884d-052f303cd8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:01.182816 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee841b3-81c5-446f-884d-052f303cd8ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:01.183918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee841b3-81c5-446f-884d-052f303cd8ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:01.185713 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee841b3-81c5-446f-884d-052f303cd8ba admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:01.199900 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1023/2041] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:01.204476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c4590-460d-4747-9f89-c8fb8634831c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:01.227638 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c4590-460d-4747-9f89-c8fb8634831c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:01.238560 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00852ff-10fc-4b16-b028-bc8e2b6d8336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:01.239659 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00852ff-10fc-4b16-b028-bc8e2b6d8336 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:01.240059 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00852ff-10fc-4b16-b028-bc8e2b6d8336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:01.240786 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00852ff-10fc-4b16-b028-bc8e2b6d8336 admin admin] PATH_INFO: `/v3/projects/8b1b954ea0cd4026b5a79ad34cca2358/users/17a81502fcf84e5695b407115e6217f0/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:01.283595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00852ff-10fc-4b16-b028-bc8e2b6d8336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b1b954ea0cd4026b5a79ad34cca2358, user_id=17a81502fcf84e5695b407115e6217f0, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:01.286350 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00852ff-10fc-4b16-b028-bc8e2b6d8336 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:01.310077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1019/2042] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:01 2022] PUT /identity/v3/projects/8b1b954ea0cd4026b5a79ad34cca2358/users/17a81502fcf84e5695b407115e6217f0/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:01.315223 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee841b3-81c5-446f-884d-052f303cd8ba admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:01.330073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee841b3-81c5-446f-884d-052f303cd8ba admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:01.421384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c741934-6aa8-4bf8-a776-da002f7f48e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:01.422644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c741934-6aa8-4bf8-a776-da002f7f48e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:01.422644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c741934-6aa8-4bf8-a776-da002f7f48e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:01.422930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c741934-6aa8-4bf8-a776-da002f7f48e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:01.423824 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c741934-6aa8-4bf8-a776-da002f7f48e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:01.427082 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c741934-6aa8-4bf8-a776-da002f7f48e2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:01.445234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1024/2043] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:01.446747 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00852ff-10fc-4b16-b028-bc8e2b6d8336 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:01.468759 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00852ff-10fc-4b16-b028-bc8e2b6d8336 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:01.480572 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e006f-bcd3-49d8-99d3-7b8c5cb0a535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:01.481496 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e006f-bcd3-49d8-99d3-7b8c5cb0a535 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:01.481785 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e006f-bcd3-49d8-99d3-7b8c5cb0a535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:01.482068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e006f-bcd3-49d8-99d3-7b8c5cb0a535 admin admin] PATH_INFO: `/v3/projects/8b1b954ea0cd4026b5a79ad34cca2358/users/17a81502fcf84e5695b407115e6217f0/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:01.487514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e006f-bcd3-49d8-99d3-7b8c5cb0a535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b1b954ea0cd4026b5a79ad34cca2358, user_id=17a81502fcf84e5695b407115e6217f0, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:01.492636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e006f-bcd3-49d8-99d3-7b8c5cb0a535 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:01.541008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a40409-e22f-4ba7-8a7e-8b8019ac86a4 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:01.541200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a40409-e22f-4ba7-8a7e-8b8019ac86a4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:01.541459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a40409-e22f-4ba7-8a7e-8b8019ac86a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:01.543574 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1020/2044] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:01 2022] PUT /identity/v3/projects/8b1b954ea0cd4026b5a79ad34cca2358/users/17a81502fcf84e5695b407115e6217f0/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:01.585849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a0a40409-e22f-4ba7-8a7e-8b8019ac86a4 None None] MFA Rules not processed for user `17a81502fcf84e5695b407115e6217f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:01.587622 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a0a40409-e22f-4ba7-8a7e-8b8019ac86a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:01.662654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:01.691960 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1025/2045] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:01.730638 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e006f-bcd3-49d8-99d3-7b8c5cb0a535 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:01.735619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ae2e006f-bcd3-49d8-99d3-7b8c5cb0a535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:01.824013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e006f-bcd3-49d8-99d3-7b8c5cb0a535 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:01.828817 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6bd57-1d10-4303-b7aa-e9a9e3a6fed6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:01.829897 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6bd57-1d10-4303-b7aa-e9a9e3a6fed6 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:01.830344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6bd57-1d10-4303-b7aa-e9a9e3a6fed6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:01.830771 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6bd57-1d10-4303-b7aa-e9a9e3a6fed6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:01.952774 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6bd57-1d10-4303-b7aa-e9a9e3a6fed6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:01.954672 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6bd57-1d10-4303-b7aa-e9a9e3a6fed6 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:01.987286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1021/2046] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:03.389490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a40409-e22f-4ba7-8a7e-8b8019ac86a4 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:03.391691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a0a40409-e22f-4ba7-8a7e-8b8019ac86a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:03.480338 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a40409-e22f-4ba7-8a7e-8b8019ac86a4 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:03.494419 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee1de3-7377-4ced-abcd-f622d48773ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:03.495346 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee1de3-7377-4ced-abcd-f622d48773ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:03.495635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee1de3-7377-4ced-abcd-f622d48773ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:03.495920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee1de3-7377-4ced-abcd-f622d48773ae admin admin] PATH_INFO: `/v3/users/ce45c850382f4cdaa3a874b2d1d8365f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:03.497601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee1de3-7377-4ced-abcd-f622d48773ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce45c850382f4cdaa3a874b2d1d8365f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:03.499737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee1de3-7377-4ced-abcd-f622d48773ae admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:03.596823 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ef611-cce9-406e-a2ed-d742aa7df979 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:03.597352 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ef611-cce9-406e-a2ed-d742aa7df979 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:03.597582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ef611-cce9-406e-a2ed-d742aa7df979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:03.603695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2eee1de3-7377-4ced-abcd-f622d48773ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce45c850382f4cdaa3a874b2d1d8365f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:03.617007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2eee1de3-7377-4ced-abcd-f622d48773ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce45c850382f4cdaa3a874b2d1d8365f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:03.619520 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2eee1de3-7377-4ced-abcd-f622d48773ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce45c850382f4cdaa3a874b2d1d8365f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:03.639631 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2eee1de3-7377-4ced-abcd-f622d48773ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce45c850382f4cdaa3a874b2d1d8365f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:03.673480 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2eee1de3-7377-4ced-abcd-f622d48773ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce45c850382f4cdaa3a874b2d1d8365f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:03.686558 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-e12ef611-cce9-406e-a2ed-d742aa7df979 None None] MFA Rules not processed for user `4a4f4d585adf40b880be44c29ecd53cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:03.688510 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e12ef611-cce9-406e-a2ed-d742aa7df979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:03.690071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1026/2047] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:03 2022] DELETE /identity/v3/users/ce45c850382f4cdaa3a874b2d1d8365f => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:03.710713 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847d610-6044-4664-89bd-791113ca75f6 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:03.710713 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847d610-6044-4664-89bd-791113ca75f6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:03.710713 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847d610-6044-4664-89bd-791113ca75f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:03.713048 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:03.760318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3847d610-6044-4664-89bd-791113ca75f6 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:03.764052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3847d610-6044-4664-89bd-791113ca75f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:03.876507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1022/2048] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:03.877535 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ef611-cce9-406e-a2ed-d742aa7df979 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:03.881012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1027/2049] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:03.881459 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e12ef611-cce9-406e-a2ed-d742aa7df979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:03.891030 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847d610-6044-4664-89bd-791113ca75f6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:03.892638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3847d610-6044-4664-89bd-791113ca75f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:03.897679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ef611-cce9-406e-a2ed-d742aa7df979 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:03.904478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ea957-1457-4e69-8ec5-0108860a5801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:03.905450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ea957-1457-4e69-8ec5-0108860a5801 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:03.905740 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ea957-1457-4e69-8ec5-0108860a5801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:03.906020 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ea957-1457-4e69-8ec5-0108860a5801 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:03.907382 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ea957-1457-4e69-8ec5-0108860a5801 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:03.910708 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ea957-1457-4e69-8ec5-0108860a5801 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:03.914913 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:03.954380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1023/2050] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:03 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:03.961945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ea957-1457-4e69-8ec5-0108860a5801 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:03.988642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847d610-6044-4664-89bd-791113ca75f6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:04.053840 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f244c15-d2e5-4192-8130-d603c679ac6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:04.054509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f244c15-d2e5-4192-8130-d603c679ac6e service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:04.054796 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f244c15-d2e5-4192-8130-d603c679ac6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:04.055076 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f244c15-d2e5-4192-8130-d603c679ac6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:04.055749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ea957-1457-4e69-8ec5-0108860a5801 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:04.062412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77393c51-538e-494e-8d85-5be29fdf3992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:04.062625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77393c51-538e-494e-8d85-5be29fdf3992 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:04.062838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77393c51-538e-494e-8d85-5be29fdf3992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:04.063672 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77393c51-538e-494e-8d85-5be29fdf3992 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:04.064406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77393c51-538e-494e-8d85-5be29fdf3992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:04.065593 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f244c15-d2e5-4192-8130-d603c679ac6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:04.067043 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77393c51-538e-494e-8d85-5be29fdf3992 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:04.067376 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f244c15-d2e5-4192-8130-d603c679ac6e service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:04.078555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1028/2051] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:25:04.096761 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1024/2052] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:03 2022] POST /identity/v3/users => generated 296 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:04.101740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f244c15-d2e5-4192-8130-d603c679ac6e service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:04.112458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f244c15-d2e5-4192-8130-d603c679ac6e service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:04.128148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975d923-2ead-44c9-9d45-90ded9f8bf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:04.128823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975d923-2ead-44c9-9d45-90ded9f8bf8a admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:04.129131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975d923-2ead-44c9-9d45-90ded9f8bf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:04.129371 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975d923-2ead-44c9-9d45-90ded9f8bf8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:04.130143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975d923-2ead-44c9-9d45-90ded9f8bf8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:04.132460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975d923-2ead-44c9-9d45-90ded9f8bf8a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:04.150895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1029/2053] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:04.151771 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77393c51-538e-494e-8d85-5be29fdf3992 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:04.164994 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77393c51-538e-494e-8d85-5be29fdf3992 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:04.170536 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520923f-da27-4ff0-a6fd-d16611ae327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:04.171546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520923f-da27-4ff0-a6fd-d16611ae327e admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:04.171853 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520923f-da27-4ff0-a6fd-d16611ae327e admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:04.172114 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520923f-da27-4ff0-a6fd-d16611ae327e admin admin] PATH_INFO: `/v3/projects/4e6e524426f1439c99af0938f0e5a6ab/users/413e322998344953a89da5bbbc12fabe/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:04.198693 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520923f-da27-4ff0-a6fd-d16611ae327e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e6e524426f1439c99af0938f0e5a6ab, user_id=413e322998344953a89da5bbbc12fabe, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:04.201406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520923f-da27-4ff0-a6fd-d16611ae327e admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:04.219498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1025/2054] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:04 2022] PUT /identity/v3/projects/4e6e524426f1439c99af0938f0e5a6ab/users/413e322998344953a89da5bbbc12fabe/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:04.223487 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975d923-2ead-44c9-9d45-90ded9f8bf8a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:04.235335 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975d923-2ead-44c9-9d45-90ded9f8bf8a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:04.313525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5d521-e0ed-4c77-9ec1-a7f21ce11bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:04.314197 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5d521-e0ed-4c77-9ec1-a7f21ce11bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:04.314486 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5d521-e0ed-4c77-9ec1-a7f21ce11bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:04.314787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5d521-e0ed-4c77-9ec1-a7f21ce11bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:04.315528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab5d521-e0ed-4c77-9ec1-a7f21ce11bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:04.317344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab5d521-e0ed-4c77-9ec1-a7f21ce11bb8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:04.331678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1030/2055] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:04.339420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520923f-da27-4ff0-a6fd-d16611ae327e admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:04.345551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520923f-da27-4ff0-a6fd-d16611ae327e admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:04.351513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f776389-ad19-461f-bc6a-1b6f61ac688d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:04.352389 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f776389-ad19-461f-bc6a-1b6f61ac688d admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:04.352717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f776389-ad19-461f-bc6a-1b6f61ac688d admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:04.352998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f776389-ad19-461f-bc6a-1b6f61ac688d admin admin] PATH_INFO: `/v3/projects/4e6e524426f1439c99af0938f0e5a6ab/users/413e322998344953a89da5bbbc12fabe/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:04.355220 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f776389-ad19-461f-bc6a-1b6f61ac688d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e6e524426f1439c99af0938f0e5a6ab, user_id=413e322998344953a89da5bbbc12fabe, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:04.357893 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f776389-ad19-461f-bc6a-1b6f61ac688d admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:04.373532 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1026/2056] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:04 2022] PUT /identity/v3/projects/4e6e524426f1439c99af0938f0e5a6ab/users/413e322998344953a89da5bbbc12fabe/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:04.421684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5d521-e0ed-4c77-9ec1-a7f21ce11bb8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:04.424704 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4ab5d521-e0ed-4c77-9ec1-a7f21ce11bb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:04.439994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5d521-e0ed-4c77-9ec1-a7f21ce11bb8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:04.516109 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:04.525242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340316f-8d98-41e4-be4a-8081ac04e4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:04.526556 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340316f-8d98-41e4-be4a-8081ac04e4c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:04.527069 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340316f-8d98-41e4-be4a-8081ac04e4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:04.527917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340316f-8d98-41e4-be4a-8081ac04e4c4 admin admin] PATH_INFO: `/v3/projects/ee028974653e4c6488746fa3ace8d15d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:04.529633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340316f-8d98-41e4-be4a-8081ac04e4c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee028974653e4c6488746fa3ace8d15d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:04.531832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340316f-8d98-41e4-be4a-8081ac04e4c4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:04.600962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7340316f-8d98-41e4-be4a-8081ac04e4c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee028974653e4c6488746fa3ace8d15d', 'request_id': 'req-7340316f-8d98-41e4-be4a-8081ac04e4c4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:04.609958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7340316f-8d98-41e4-be4a-8081ac04e4c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee028974653e4c6488746fa3ace8d15d', 'request_id': 'req-7340316f-8d98-41e4-be4a-8081ac04e4c4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:04.620912 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7340316f-8d98-41e4-be4a-8081ac04e4c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee028974653e4c6488746fa3ace8d15d', 'request_id': 'req-7340316f-8d98-41e4-be4a-8081ac04e4c4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:04.640322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7340316f-8d98-41e4-be4a-8081ac04e4c4 admin admin] The token cache is being invalidate because project ee028974653e4c6488746fa3ace8d15d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:04.641404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7340316f-8d98-41e4-be4a-8081ac04e4c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:04.642455 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7340316f-8d98-41e4-be4a-8081ac04e4c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:04.649875 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1031/2057] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:04 2022] DELETE /identity/v3/projects/ee028974653e4c6488746fa3ace8d15d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:04.654208 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f776389-ad19-461f-bc6a-1b6f61ac688d admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:04.658742 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0f776389-ad19-461f-bc6a-1b6f61ac688d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:04.698841 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f776389-ad19-461f-bc6a-1b6f61ac688d admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:04.800954 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:04.823402 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5eecc-0695-4021-9537-1c9c9252f79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:04.825717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5eecc-0695-4021-9537-1c9c9252f79a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:04.826085 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5eecc-0695-4021-9537-1c9c9252f79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:04.827235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5eecc-0695-4021-9537-1c9c9252f79a admin admin] PATH_INFO: `/v3/users/0c5ee88cd213478697c040ef84b4a1d2` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:04.831697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5eecc-0695-4021-9537-1c9c9252f79a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c5ee88cd213478697c040ef84b4a1d2)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:04.835450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5eecc-0695-4021-9537-1c9c9252f79a admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:04.912673 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e9b5eecc-0695-4021-9537-1c9c9252f79a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c5ee88cd213478697c040ef84b4a1d2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:04.973116 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e9b5eecc-0695-4021-9537-1c9c9252f79a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c5ee88cd213478697c040ef84b4a1d2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:04.996517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340316f-8d98-41e4-be4a-8081ac04e4c4 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:04.998153 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7340316f-8d98-41e4-be4a-8081ac04e4c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:05.000392 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e9b5eecc-0695-4021-9537-1c9c9252f79a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c5ee88cd213478697c040ef84b4a1d2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:05.011165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e9b5eecc-0695-4021-9537-1c9c9252f79a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c5ee88cd213478697c040ef84b4a1d2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:05.012256 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e9b5eecc-0695-4021-9537-1c9c9252f79a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c5ee88cd213478697c040ef84b4a1d2'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:05.019721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1027/2058] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:04 2022] DELETE /identity/v3/users/0c5ee88cd213478697c040ef84b4a1d2 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:05.021162 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340316f-8d98-41e4-be4a-8081ac04e4c4 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:05.139688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162dab6-bc2f-4a55-b92e-ad60c1dc50ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:05.140657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162dab6-bc2f-4a55-b92e-ad60c1dc50ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:05.140946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162dab6-bc2f-4a55-b92e-ad60c1dc50ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:05.141270 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162dab6-bc2f-4a55-b92e-ad60c1dc50ae admin admin] PATH_INFO: `/v3/users/bb8c4ec33399483392d80fb82a6d50eb` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:05.144701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162dab6-bc2f-4a55-b92e-ad60c1dc50ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb8c4ec33399483392d80fb82a6d50eb)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:05.149479 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162dab6-bc2f-4a55-b92e-ad60c1dc50ae admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:05.245562 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3162dab6-bc2f-4a55-b92e-ad60c1dc50ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb8c4ec33399483392d80fb82a6d50eb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:05.256553 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3162dab6-bc2f-4a55-b92e-ad60c1dc50ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb8c4ec33399483392d80fb82a6d50eb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:05.257251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3162dab6-bc2f-4a55-b92e-ad60c1dc50ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb8c4ec33399483392d80fb82a6d50eb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:05.278666 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3162dab6-bc2f-4a55-b92e-ad60c1dc50ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb8c4ec33399483392d80fb82a6d50eb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:05.300815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3162dab6-bc2f-4a55-b92e-ad60c1dc50ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb8c4ec33399483392d80fb82a6d50eb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:05.318404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1032/2059] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:04 2022] DELETE /identity/v3/users/bb8c4ec33399483392d80fb82a6d50eb => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:05.320360 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d6ea1-883b-4098-8078-94feebe9e647 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:05.320705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d6ea1-883b-4098-8078-94feebe9e647 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:05.321102 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d6ea1-883b-4098-8078-94feebe9e647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:05.350484 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-c88d6ea1-883b-4098-8078-94feebe9e647 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:05.351813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c88d6ea1-883b-4098-8078-94feebe9e647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:05.477007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1028/2060] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:05.503688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162dab6-bc2f-4a55-b92e-ad60c1dc50ae admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:05.505193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3162dab6-bc2f-4a55-b92e-ad60c1dc50ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:05.520133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:05.615645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162dab6-bc2f-4a55-b92e-ad60c1dc50ae admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:05.638223 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97432bde-fa2a-4eec-850e-e4678e7c26c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:05.638901 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97432bde-fa2a-4eec-850e-e4678e7c26c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:05.639194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97432bde-fa2a-4eec-850e-e4678e7c26c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:05.639469 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97432bde-fa2a-4eec-850e-e4678e7c26c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:05.674853 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97432bde-fa2a-4eec-850e-e4678e7c26c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:05.676428 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97432bde-fa2a-4eec-850e-e4678e7c26c0 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:05.690120 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1033/2061] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:05.805749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d6ea1-883b-4098-8078-94feebe9e647 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:05.808122 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c88d6ea1-883b-4098-8078-94feebe9e647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:05.852435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d6ea1-883b-4098-8078-94feebe9e647 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:05.870624 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace2058-79ca-411a-a70b-7a1ef9161289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:05.879696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace2058-79ca-411a-a70b-7a1ef9161289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:05.879696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace2058-79ca-411a-a70b-7a1ef9161289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:05.879696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace2058-79ca-411a-a70b-7a1ef9161289 admin admin] PATH_INFO: `/v3/projects/38318668926943bab6bd1aa121ffd503` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:05.879696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace2058-79ca-411a-a70b-7a1ef9161289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38318668926943bab6bd1aa121ffd503)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:05.898823 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace2058-79ca-411a-a70b-7a1ef9161289 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:06.028166 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9ace2058-79ca-411a-a70b-7a1ef9161289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38318668926943bab6bd1aa121ffd503', 'request_id': 'req-9ace2058-79ca-411a-a70b-7a1ef9161289'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.054376 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9ace2058-79ca-411a-a70b-7a1ef9161289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38318668926943bab6bd1aa121ffd503', 'request_id': 'req-9ace2058-79ca-411a-a70b-7a1ef9161289'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.059591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:06.066875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9ace2058-79ca-411a-a70b-7a1ef9161289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38318668926943bab6bd1aa121ffd503', 'request_id': 'req-9ace2058-79ca-411a-a70b-7a1ef9161289'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.088280 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9ace2058-79ca-411a-a70b-7a1ef9161289 admin admin] The token cache is being invalidate because project 38318668926943bab6bd1aa121ffd503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:06.088883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9ace2058-79ca-411a-a70b-7a1ef9161289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.090542 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9ace2058-79ca-411a-a70b-7a1ef9161289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.104268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97432bde-fa2a-4eec-850e-e4678e7c26c0 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:06.104553 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1029/2062] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:05 2022] DELETE /identity/v3/projects/38318668926943bab6bd1aa121ffd503 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:06.108204 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-97432bde-fa2a-4eec-850e-e4678e7c26c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:06.111734 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace2058-79ca-411a-a70b-7a1ef9161289 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:06.114675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-9ace2058-79ca-411a-a70b-7a1ef9161289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:06.134464 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97432bde-fa2a-4eec-850e-e4678e7c26c0 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:06.264526 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace2058-79ca-411a-a70b-7a1ef9161289 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:06.266917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc365d18-46b2-4067-9a42-15b8759893f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:06.268127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc365d18-46b2-4067-9a42-15b8759893f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:06.268452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc365d18-46b2-4067-9a42-15b8759893f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:06.268734 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc365d18-46b2-4067-9a42-15b8759893f1 admin admin] PATH_INFO: `/v3/projects/4c12e93a21a8436db1bb7276b1d3b8f6` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:06.269998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc365d18-46b2-4067-9a42-15b8759893f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c12e93a21a8436db1bb7276b1d3b8f6)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:06.272075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd638bd7-c949-4c8f-8c7c-63575cdec057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:06.272613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc365d18-46b2-4067-9a42-15b8759893f1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:06.273124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd638bd7-c949-4c8f-8c7c-63575cdec057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:06.273755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd638bd7-c949-4c8f-8c7c-63575cdec057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:06.274043 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd638bd7-c949-4c8f-8c7c-63575cdec057 admin admin] PATH_INFO: `/v3/users/0ed78e5708ff402db886b2bd62d9e4ce` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:06.275624 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd638bd7-c949-4c8f-8c7c-63575cdec057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ed78e5708ff402db886b2bd62d9e4ce)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:06.280032 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd638bd7-c949-4c8f-8c7c-63575cdec057 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:06.334118 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fc365d18-46b2-4067-9a42-15b8759893f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c12e93a21a8436db1bb7276b1d3b8f6', 'request_id': 'req-fc365d18-46b2-4067-9a42-15b8759893f1'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.341816 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fc365d18-46b2-4067-9a42-15b8759893f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c12e93a21a8436db1bb7276b1d3b8f6', 'request_id': 'req-fc365d18-46b2-4067-9a42-15b8759893f1'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.376836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-dd638bd7-c949-4c8f-8c7c-63575cdec057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ed78e5708ff402db886b2bd62d9e4ce'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.380243 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fc365d18-46b2-4067-9a42-15b8759893f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c12e93a21a8436db1bb7276b1d3b8f6', 'request_id': 'req-fc365d18-46b2-4067-9a42-15b8759893f1'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.391273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-dd638bd7-c949-4c8f-8c7c-63575cdec057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ed78e5708ff402db886b2bd62d9e4ce'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.401019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fc365d18-46b2-4067-9a42-15b8759893f1 admin admin] The token cache is being invalidate because project 4c12e93a21a8436db1bb7276b1d3b8f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:06.401312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fc365d18-46b2-4067-9a42-15b8759893f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.402146 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fc365d18-46b2-4067-9a42-15b8759893f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.405545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1034/2063] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:06 2022] DELETE /identity/v3/projects/4c12e93a21a8436db1bb7276b1d3b8f6 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:06.409594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc365d18-46b2-4067-9a42-15b8759893f1 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:06.412808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fc365d18-46b2-4067-9a42-15b8759893f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:06.427699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-dd638bd7-c949-4c8f-8c7c-63575cdec057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ed78e5708ff402db886b2bd62d9e4ce'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.441965 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-dd638bd7-c949-4c8f-8c7c-63575cdec057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ed78e5708ff402db886b2bd62d9e4ce'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.442915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-dd638bd7-c949-4c8f-8c7c-63575cdec057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ed78e5708ff402db886b2bd62d9e4ce'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.452109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1030/2064] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:06 2022] DELETE /identity/v3/users/0ed78e5708ff402db886b2bd62d9e4ce => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:06.484064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:06.523840 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc365d18-46b2-4067-9a42-15b8759893f1 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:06.696798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1ee56-500d-4059-9d32-b178de189233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:06.699872 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1ee56-500d-4059-9d32-b178de189233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:06.701060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1ee56-500d-4059-9d32-b178de189233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:06.703515 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1ee56-500d-4059-9d32-b178de189233 admin admin] PATH_INFO: `/v3/users/17a81502fcf84e5695b407115e6217f0` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:06.707428 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1ee56-500d-4059-9d32-b178de189233 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17a81502fcf84e5695b407115e6217f0)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:06.710197 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1ee56-500d-4059-9d32-b178de189233 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:06.844499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a2a1ee56-500d-4059-9d32-b178de189233 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17a81502fcf84e5695b407115e6217f0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.866573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a2a1ee56-500d-4059-9d32-b178de189233 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17a81502fcf84e5695b407115e6217f0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.867617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a2a1ee56-500d-4059-9d32-b178de189233 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17a81502fcf84e5695b407115e6217f0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.882199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a2a1ee56-500d-4059-9d32-b178de189233 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17a81502fcf84e5695b407115e6217f0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.919791 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a2a1ee56-500d-4059-9d32-b178de189233 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17a81502fcf84e5695b407115e6217f0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:06.940146 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1035/2065] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:06 2022] DELETE /identity/v3/users/17a81502fcf84e5695b407115e6217f0 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:06.999400 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd638bd7-c949-4c8f-8c7c-63575cdec057 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:07.001183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-dd638bd7-c949-4c8f-8c7c-63575cdec057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:07.023206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd638bd7-c949-4c8f-8c7c-63575cdec057 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:07.114346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa348ba-79aa-48eb-8cab-51987bfe8525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:07.115951 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa348ba-79aa-48eb-8cab-51987bfe8525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:07.116369 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa348ba-79aa-48eb-8cab-51987bfe8525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:07.117384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa348ba-79aa-48eb-8cab-51987bfe8525 admin admin] PATH_INFO: `/v3/projects/52e44c059a894a28ad60514bc6ddfe15` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:07.118799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa348ba-79aa-48eb-8cab-51987bfe8525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52e44c059a894a28ad60514bc6ddfe15)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:07.123307 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa348ba-79aa-48eb-8cab-51987bfe8525 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:07.137293 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:07.203851 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-caa348ba-79aa-48eb-8cab-51987bfe8525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52e44c059a894a28ad60514bc6ddfe15', 'request_id': 'req-caa348ba-79aa-48eb-8cab-51987bfe8525'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:07.217021 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-caa348ba-79aa-48eb-8cab-51987bfe8525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52e44c059a894a28ad60514bc6ddfe15', 'request_id': 'req-caa348ba-79aa-48eb-8cab-51987bfe8525'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:07.237739 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-caa348ba-79aa-48eb-8cab-51987bfe8525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52e44c059a894a28ad60514bc6ddfe15', 'request_id': 'req-caa348ba-79aa-48eb-8cab-51987bfe8525'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:07.265745 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-caa348ba-79aa-48eb-8cab-51987bfe8525 admin admin] The token cache is being invalidate because project 52e44c059a894a28ad60514bc6ddfe15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:07.265961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-caa348ba-79aa-48eb-8cab-51987bfe8525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:07.267740 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-caa348ba-79aa-48eb-8cab-51987bfe8525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:07.270204 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1031/2066] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:06 2022] DELETE /identity/v3/projects/52e44c059a894a28ad60514bc6ddfe15 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:07.276666 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514febe-cbe1-4517-88a1-7e51f96153b3 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:07.277261 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514febe-cbe1-4517-88a1-7e51f96153b3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:07.277628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514febe-cbe1-4517-88a1-7e51f96153b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:07.302925 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d514febe-cbe1-4517-88a1-7e51f96153b3 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:07.304157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d514febe-cbe1-4517-88a1-7e51f96153b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:07.447783 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1036/2067] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:07.457072 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2101977-f895-43b8-bea9-7a2576371ac0 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:07.457428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2101977-f895-43b8-bea9-7a2576371ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:07.457644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2101977-f895-43b8-bea9-7a2576371ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:07.479210 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-f2101977-f895-43b8-bea9-7a2576371ac0 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:07.484419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f2101977-f895-43b8-bea9-7a2576371ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:07.494354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1032/2068] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:07.498337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514febe-cbe1-4517-88a1-7e51f96153b3 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:07.507557 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:07.513131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514febe-cbe1-4517-88a1-7e51f96153b3 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:07.524943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a7e56-1d43-4b9c-bd41-3ac38a1dd364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:07.525891 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a7e56-1d43-4b9c-bd41-3ac38a1dd364 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:07.525891 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a7e56-1d43-4b9c-bd41-3ac38a1dd364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:07.525891 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a7e56-1d43-4b9c-bd41-3ac38a1dd364 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:07.527322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a7e56-1d43-4b9c-bd41-3ac38a1dd364 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:07.529257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a7e56-1d43-4b9c-bd41-3ac38a1dd364 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:07.530564 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a7e56-1d43-4b9c-bd41-3ac38a1dd364 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:07.551210 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1037/2069] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:25:07.556939 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2101977-f895-43b8-bea9-7a2576371ac0 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:07.577056 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2101977-f895-43b8-bea9-7a2576371ac0 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:07.580486 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95602e8b-2c35-457f-9901-8524c9756fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:07.581451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95602e8b-2c35-457f-9901-8524c9756fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:07.581451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95602e8b-2c35-457f-9901-8524c9756fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:07.581757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95602e8b-2c35-457f-9901-8524c9756fd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:07.583038 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95602e8b-2c35-457f-9901-8524c9756fd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:07.586236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95602e8b-2c35-457f-9901-8524c9756fd6 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:07.668793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1033/2070] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:07 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:07.669967 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a7e56-1d43-4b9c-bd41-3ac38a1dd364 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:07.686953 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a7e56-1d43-4b9c-bd41-3ac38a1dd364 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:07.753956 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95602e8b-2c35-457f-9901-8524c9756fd6 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:07.755808 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-95602e8b-2c35-457f-9901-8524c9756fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:07.889435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78638e-b10e-4491-bf0c-1d6e8130bed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:07.890134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78638e-b10e-4491-bf0c-1d6e8130bed6 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:07.890545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78638e-b10e-4491-bf0c-1d6e8130bed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:07.890847 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78638e-b10e-4491-bf0c-1d6e8130bed6 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:07.891409 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78638e-b10e-4491-bf0c-1d6e8130bed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:07.893428 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78638e-b10e-4491-bf0c-1d6e8130bed6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:07.969657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1038/2071] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:07 2022] POST /identity/v3/users => generated 287 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:07.970709 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78638e-b10e-4491-bf0c-1d6e8130bed6 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:07.972874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95602e8b-2c35-457f-9901-8524c9756fd6 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:07.991476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78638e-b10e-4491-bf0c-1d6e8130bed6 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:08.007219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a714f-309a-4b6c-8893-2679fe136639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:08.008023 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a714f-309a-4b6c-8893-2679fe136639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:08.011572 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a714f-309a-4b6c-8893-2679fe136639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:08.012355 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b128b1-f692-4d0b-8768-7665a157962f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:08.013215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b128b1-f692-4d0b-8768-7665a157962f admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:08.015567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a714f-309a-4b6c-8893-2679fe136639 admin admin] PATH_INFO: `/v3/projects/8b1b954ea0cd4026b5a79ad34cca2358` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:08.016094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b128b1-f692-4d0b-8768-7665a157962f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:08.016561 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b128b1-f692-4d0b-8768-7665a157962f admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:08.020004 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b128b1-f692-4d0b-8768-7665a157962f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:08.025729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a714f-309a-4b6c-8893-2679fe136639 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b1b954ea0cd4026b5a79ad34cca2358)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:08.026479 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b128b1-f692-4d0b-8768-7665a157962f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:08.031579 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a714f-309a-4b6c-8893-2679fe136639 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:08.100102 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1039/2072] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:08.101184 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b128b1-f692-4d0b-8768-7665a157962f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:08.168870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b128b1-f692-4d0b-8768-7665a157962f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:08.195527 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b38a714f-309a-4b6c-8893-2679fe136639 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b1b954ea0cd4026b5a79ad34cca2358', 'request_id': 'req-b38a714f-309a-4b6c-8893-2679fe136639'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:08.209052 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b38a714f-309a-4b6c-8893-2679fe136639 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b1b954ea0cd4026b5a79ad34cca2358', 'request_id': 'req-b38a714f-309a-4b6c-8893-2679fe136639'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:08.219245 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b38a714f-309a-4b6c-8893-2679fe136639 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b1b954ea0cd4026b5a79ad34cca2358', 'request_id': 'req-b38a714f-309a-4b6c-8893-2679fe136639'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:08.238666 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:08.250830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b38a714f-309a-4b6c-8893-2679fe136639 admin admin] The token cache is being invalidate because project 8b1b954ea0cd4026b5a79ad34cca2358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:08.251357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b38a714f-309a-4b6c-8893-2679fe136639 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:08.254017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b38a714f-309a-4b6c-8893-2679fe136639 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:08.257799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1034/2073] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:07 2022] DELETE /identity/v3/projects/8b1b954ea0cd4026b5a79ad34cca2358 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:08.266584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f992b-3ccd-4029-a357-fe75df088f85 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:08.267079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f992b-3ccd-4029-a357-fe75df088f85 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:08.267504 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f992b-3ccd-4029-a357-fe75df088f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:08.276554 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72861b9f-cf9a-4b2a-be9a-991a6aad6a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:08.277240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72861b9f-cf9a-4b2a-be9a-991a6aad6a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:08.277523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72861b9f-cf9a-4b2a-be9a-991a6aad6a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:08.277807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72861b9f-cf9a-4b2a-be9a-991a6aad6a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:08.278527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72861b9f-cf9a-4b2a-be9a-991a6aad6a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:08.280140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72861b9f-cf9a-4b2a-be9a-991a6aad6a59 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:08.291269 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1040/2074] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:08.296153 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72861b9f-cf9a-4b2a-be9a-991a6aad6a59 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:08.298275 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-630f992b-3ccd-4029-a357-fe75df088f85 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:08.299786 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-630f992b-3ccd-4029-a357-fe75df088f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:08.301647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-72861b9f-cf9a-4b2a-be9a-991a6aad6a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:08.321915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1035/2075] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:08.331862 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896fe00-ef2f-462f-95fa-74081217da18 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:08.332720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896fe00-ef2f-462f-95fa-74081217da18 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:08.334719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896fe00-ef2f-462f-95fa-74081217da18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:08.355449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72861b9f-cf9a-4b2a-be9a-991a6aad6a59 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:08.371839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841c252-6418-4302-b6fa-db1be78d0168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:08.385437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841c252-6418-4302-b6fa-db1be78d0168 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:08.385983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841c252-6418-4302-b6fa-db1be78d0168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:08.386499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841c252-6418-4302-b6fa-db1be78d0168 admin admin] PATH_INFO: `/v3/projects/aba8221c6e144e94ae026555d5317576/users/83ba05eb66ec44598ec0809cfc0c4ccd/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:08.408131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841c252-6418-4302-b6fa-db1be78d0168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba8221c6e144e94ae026555d5317576, user_id=83ba05eb66ec44598ec0809cfc0c4ccd, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:08.411494 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841c252-6418-4302-b6fa-db1be78d0168 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:08.418160 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4896fe00-ef2f-462f-95fa-74081217da18 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:08.422072 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4896fe00-ef2f-462f-95fa-74081217da18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:08.440651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1041/2076] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:08 2022] PUT /identity/v3/projects/aba8221c6e144e94ae026555d5317576/users/83ba05eb66ec44598ec0809cfc0c4ccd/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:08.447364 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d1840-9213-40b2-a468-ff3e292bdd11 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:08.447830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d1840-9213-40b2-a468-ff3e292bdd11 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:08.448189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d1840-9213-40b2-a468-ff3e292bdd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:08.476570 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-cb1d1840-9213-40b2-a468-ff3e292bdd11 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:08.478098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cb1d1840-9213-40b2-a468-ff3e292bdd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:08.548484 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:08.556541 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1036/2077] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:08.560690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896fe00-ef2f-462f-95fa-74081217da18 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:08.575779 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1042/2078] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:08.576921 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896fe00-ef2f-462f-95fa-74081217da18 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:08.587172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3916d-385e-4151-ba0a-1ef8fb3315d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:08.588080 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d1840-9213-40b2-a468-ff3e292bdd11 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:08.588598 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3916d-385e-4151-ba0a-1ef8fb3315d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:08.589228 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3916d-385e-4151-ba0a-1ef8fb3315d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:08.589656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3916d-385e-4151-ba0a-1ef8fb3315d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:08.590465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3916d-385e-4151-ba0a-1ef8fb3315d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:08.590789 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3916d-385e-4151-ba0a-1ef8fb3315d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:08.592460 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3916d-385e-4151-ba0a-1ef8fb3315d8 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:08.594700 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cb1d1840-9213-40b2-a468-ff3e292bdd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:08.625317 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1037/2079] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:25:08.632180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3916d-385e-4151-ba0a-1ef8fb3315d8 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:08.649112 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3916d-385e-4151-ba0a-1ef8fb3315d8 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:08.655682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c4ea6-4d0f-4910-a5d6-1e37bc65f03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:08.656685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c4ea6-4d0f-4910-a5d6-1e37bc65f03a admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:08.657039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c4ea6-4d0f-4910-a5d6-1e37bc65f03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:08.657383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c4ea6-4d0f-4910-a5d6-1e37bc65f03a admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:08.658153 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c4ea6-4d0f-4910-a5d6-1e37bc65f03a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:08.660163 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c4ea6-4d0f-4910-a5d6-1e37bc65f03a admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:08.681492 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d1840-9213-40b2-a468-ff3e292bdd11 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:08.689310 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33945a-82d9-47c3-be10-70beb47114a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:08.689526 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33945a-82d9-47c3-be10-70beb47114a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:08.689658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33945a-82d9-47c3-be10-70beb47114a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:08.689929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33945a-82d9-47c3-be10-70beb47114a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:08.696649 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1038/2080] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:08 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:08.701957 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c4ea6-4d0f-4910-a5d6-1e37bc65f03a admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:08.702658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33945a-82d9-47c3-be10-70beb47114a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:08.704848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33945a-82d9-47c3-be10-70beb47114a2 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:08.758423 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1043/2081] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:08.878783 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c4ea6-4d0f-4910-a5d6-1e37bc65f03a admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:08.886651 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293655b-cfdc-4c67-8bad-adc2504b565f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:08.887847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293655b-cfdc-4c67-8bad-adc2504b565f admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:08.888137 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293655b-cfdc-4c67-8bad-adc2504b565f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:08.888799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293655b-cfdc-4c67-8bad-adc2504b565f admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:08.889919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293655b-cfdc-4c67-8bad-adc2504b565f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:08.892591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293655b-cfdc-4c67-8bad-adc2504b565f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:08.927322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1039/2082] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:08 2022] POST /identity/v3/users => generated 288 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:08.932503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33945a-82d9-47c3-be10-70beb47114a2 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:08.950723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33945a-82d9-47c3-be10-70beb47114a2 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:08.963962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd506c2-983e-4995-8ac0-4f6da5807c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:08.966460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd506c2-983e-4995-8ac0-4f6da5807c73 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:08.966995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd506c2-983e-4995-8ac0-4f6da5807c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:08.967788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd506c2-983e-4995-8ac0-4f6da5807c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:08.970159 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd506c2-983e-4995-8ac0-4f6da5807c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:08.973286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd506c2-983e-4995-8ac0-4f6da5807c73 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:08.994468 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1044/2083] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:08.999263 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293655b-cfdc-4c67-8bad-adc2504b565f admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:09.023059 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293655b-cfdc-4c67-8bad-adc2504b565f admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:09.044402 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c755a69-c158-4487-9439-5bb6ded9c1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:09.045462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c755a69-c158-4487-9439-5bb6ded9c1cb admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:09.045978 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c755a69-c158-4487-9439-5bb6ded9c1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:09.046433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c755a69-c158-4487-9439-5bb6ded9c1cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:09.047364 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c755a69-c158-4487-9439-5bb6ded9c1cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:09.049456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c755a69-c158-4487-9439-5bb6ded9c1cb admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:09.079780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1040/2084] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:09.083904 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd506c2-983e-4995-8ac0-4f6da5807c73 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:09.100762 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd506c2-983e-4995-8ac0-4f6da5807c73 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:09.116833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c09367-8a52-4edb-933c-d60c1b2869fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:09.118495 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c09367-8a52-4edb-933c-d60c1b2869fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:09.119330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c09367-8a52-4edb-933c-d60c1b2869fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:09.119633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c09367-8a52-4edb-933c-d60c1b2869fc admin admin] PATH_INFO: `/v3/projects/a611e0777d8d43b58895bd584509c1f2/users/9e45c027745646fc865f0ee5e44266f1/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:09.182987 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c09367-8a52-4edb-933c-d60c1b2869fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a611e0777d8d43b58895bd584509c1f2, user_id=9e45c027745646fc865f0ee5e44266f1, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:09.195885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c09367-8a52-4edb-933c-d60c1b2869fc admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:09.228943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1045/2085] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:09 2022] PUT /identity/v3/projects/a611e0777d8d43b58895bd584509c1f2/users/9e45c027745646fc865f0ee5e44266f1/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:09.231159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70885cb5-c6a4-4e2d-9fd9-bace11c95ad9 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:09.231672 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70885cb5-c6a4-4e2d-9fd9-bace11c95ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:09.232183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70885cb5-c6a4-4e2d-9fd9-bace11c95ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:09.291705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-70885cb5-c6a4-4e2d-9fd9-bace11c95ad9 None None] MFA Rules not processed for user `9e45c027745646fc865f0ee5e44266f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:09.293597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-70885cb5-c6a4-4e2d-9fd9-bace11c95ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:09.355799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:09.504446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1041/2086] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:09.513776 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c09367-8a52-4edb-933c-d60c1b2869fc admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:09.515795 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f2c09367-8a52-4edb-933c-d60c1b2869fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:09.643662 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c09367-8a52-4edb-933c-d60c1b2869fc admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:09.647962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d302d3a-e248-4384-8951-07c465fce018 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:09.648835 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d302d3a-e248-4384-8951-07c465fce018 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:09.649191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d302d3a-e248-4384-8951-07c465fce018 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:09.649554 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d302d3a-e248-4384-8951-07c465fce018 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:09.672048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d302d3a-e248-4384-8951-07c465fce018 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:09.673243 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d302d3a-e248-4384-8951-07c465fce018 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:09.696047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1046/2087] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:11.231376 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69a474-6a75-4b81-96b8-41054b40cefd None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:11.232023 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69a474-6a75-4b81-96b8-41054b40cefd None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:11.232023 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69a474-6a75-4b81-96b8-41054b40cefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:11.291218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-2f69a474-6a75-4b81-96b8-41054b40cefd None None] MFA Rules not processed for user `83ba05eb66ec44598ec0809cfc0c4ccd`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:11.294351 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-2f69a474-6a75-4b81-96b8-41054b40cefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:11.394529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1042/2088] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:11.410761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d302d3a-e248-4384-8951-07c465fce018 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:11.442448 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d302d3a-e248-4384-8951-07c465fce018 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:11.457596 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81cca22-d953-4175-b480-36ca7a719ea5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:11.459005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cca22-d953-4175-b480-36ca7a719ea5 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:11.459294 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cca22-d953-4175-b480-36ca7a719ea5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:11.459855 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cca22-d953-4175-b480-36ca7a719ea5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:11.480827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81cca22-d953-4175-b480-36ca7a719ea5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:11.482437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81cca22-d953-4175-b480-36ca7a719ea5 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:11.506315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1047/2089] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:13.221757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e52088-02d3-4f39-be90-6a58ec3367be None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:13.222631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e52088-02d3-4f39-be90-6a58ec3367be None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:13.222910 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e52088-02d3-4f39-be90-6a58ec3367be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:13.242993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-15e52088-02d3-4f39-be90-6a58ec3367be None None] MFA Rules not processed for user `83ba05eb66ec44598ec0809cfc0c4ccd`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:13.244213 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-15e52088-02d3-4f39-be90-6a58ec3367be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:13.251209 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1043/2090] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:14.259923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baca09e-6fdd-4fa4-953a-d19d7d8b4b1e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:14.260868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baca09e-6fdd-4fa4-953a-d19d7d8b4b1e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:14.261156 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baca09e-6fdd-4fa4-953a-d19d7d8b4b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:14.290571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-6baca09e-6fdd-4fa4-953a-d19d7d8b4b1e None None] MFA Rules not processed for user `413e322998344953a89da5bbbc12fabe`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:14.291852 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6baca09e-6fdd-4fa4-953a-d19d7d8b4b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:14.328533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:14.388000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1048/2091] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:14.406644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e52088-02d3-4f39-be90-6a58ec3367be None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:14.409163 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-15e52088-02d3-4f39-be90-6a58ec3367be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:14.507753 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e52088-02d3-4f39-be90-6a58ec3367be None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:14.514049 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd6fa5-b870-4092-aeff-9387a63c9761 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:14.514873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd6fa5-b870-4092-aeff-9387a63c9761 service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:14.515264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd6fa5-b870-4092-aeff-9387a63c9761 service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:14.515644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd6fa5-b870-4092-aeff-9387a63c9761 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:14.517830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-a8dd6fa5-b870-4092-aeff-9387a63c9761 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:14.580577 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:14.605297 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd6fa5-b870-4092-aeff-9387a63c9761 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:14.607275 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd6fa5-b870-4092-aeff-9387a63c9761 service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:14.622980 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1044/2092] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:18.587962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baca09e-6fdd-4fa4-953a-d19d7d8b4b1e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:18.589918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6baca09e-6fdd-4fa4-953a-d19d7d8b4b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:18.716425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baca09e-6fdd-4fa4-953a-d19d7d8b4b1e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:18.735914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec605e-3faa-4c26-8ce1-543d5d56c5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:18.737499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec605e-3faa-4c26-8ce1-543d5d56c5d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:18.738449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec605e-3faa-4c26-8ce1-543d5d56c5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:18.738967 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec605e-3faa-4c26-8ce1-543d5d56c5d2 admin admin] PATH_INFO: `/v3/users/3b7b1014c466482b95820616b54dd13a` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:18.740467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec605e-3faa-4c26-8ce1-543d5d56c5d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b7b1014c466482b95820616b54dd13a)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:18.760507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec605e-3faa-4c26-8ce1-543d5d56c5d2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:18.904801 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a3ec605e-3faa-4c26-8ce1-543d5d56c5d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b7b1014c466482b95820616b54dd13a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:18.922782 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a3ec605e-3faa-4c26-8ce1-543d5d56c5d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b7b1014c466482b95820616b54dd13a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:18.923950 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a3ec605e-3faa-4c26-8ce1-543d5d56c5d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b7b1014c466482b95820616b54dd13a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:18.939587 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a3ec605e-3faa-4c26-8ce1-543d5d56c5d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b7b1014c466482b95820616b54dd13a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:18.959624 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a3ec605e-3faa-4c26-8ce1-543d5d56c5d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b7b1014c466482b95820616b54dd13a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:18.980427 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1049/2093] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:18 2022] DELETE /identity/v3/users/3b7b1014c466482b95820616b54dd13a => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:19.265581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd6fa5-b870-4092-aeff-9387a63c9761 service nova] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:19.279561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd6fa5-b870-4092-aeff-9387a63c9761 service nova] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:19.395650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ab591-b914-428c-8d5c-3d728f8475b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:19.397161 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ab591-b914-428c-8d5c-3d728f8475b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:19.397451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ab591-b914-428c-8d5c-3d728f8475b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:19.397789 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ab591-b914-428c-8d5c-3d728f8475b8 admin admin] PATH_INFO: `/v3/projects/5231dfdf49c44f7ab191159ea563dad7` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:19.399267 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ab591-b914-428c-8d5c-3d728f8475b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5231dfdf49c44f7ab191159ea563dad7)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:19.404517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ab591-b914-428c-8d5c-3d728f8475b8 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:19.487399 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-008ab591-b914-428c-8d5c-3d728f8475b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5231dfdf49c44f7ab191159ea563dad7', 'request_id': 'req-008ab591-b914-428c-8d5c-3d728f8475b8'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:19.498693 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-008ab591-b914-428c-8d5c-3d728f8475b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5231dfdf49c44f7ab191159ea563dad7', 'request_id': 'req-008ab591-b914-428c-8d5c-3d728f8475b8'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:19.508570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-008ab591-b914-428c-8d5c-3d728f8475b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5231dfdf49c44f7ab191159ea563dad7', 'request_id': 'req-008ab591-b914-428c-8d5c-3d728f8475b8'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:19.538250 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-008ab591-b914-428c-8d5c-3d728f8475b8 admin admin] The token cache is being invalidate because project 5231dfdf49c44f7ab191159ea563dad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:19.538581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-008ab591-b914-428c-8d5c-3d728f8475b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:19.539319 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-008ab591-b914-428c-8d5c-3d728f8475b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:19.541174 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1045/2094] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:19 2022] DELETE /identity/v3/projects/5231dfdf49c44f7ab191159ea563dad7 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:19.546243 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f1a82-f27d-4611-9bb5-df52a7cf6934 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:19.546548 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f1a82-f27d-4611-9bb5-df52a7cf6934 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:19.546823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f1a82-f27d-4611-9bb5-df52a7cf6934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:19.567034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-385f1a82-f27d-4611-9bb5-df52a7cf6934 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:19.568958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-385f1a82-f27d-4611-9bb5-df52a7cf6934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:19.622258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:19.651907 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1050/2095] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:19.660562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240f850-38c1-40dc-819a-c46dc17f18f3 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:19.660855 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240f850-38c1-40dc-819a-c46dc17f18f3 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:19.661140 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240f850-38c1-40dc-819a-c46dc17f18f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:19.680590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-6240f850-38c1-40dc-819a-c46dc17f18f3 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:19.681881 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6240f850-38c1-40dc-819a-c46dc17f18f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:19.690484 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1046/2096] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:19.695250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f1a82-f27d-4611-9bb5-df52a7cf6934 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:19.705393 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f1a82-f27d-4611-9bb5-df52a7cf6934 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:19.710850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d563391-a3c6-4d00-90fc-1ff1c4747194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:19.711644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d563391-a3c6-4d00-90fc-1ff1c4747194 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:19.711936 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d563391-a3c6-4d00-90fc-1ff1c4747194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:19.712227 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d563391-a3c6-4d00-90fc-1ff1c4747194 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:19.713022 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d563391-a3c6-4d00-90fc-1ff1c4747194 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:19.713295 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d563391-a3c6-4d00-90fc-1ff1c4747194 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:19.714924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d563391-a3c6-4d00-90fc-1ff1c4747194 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:19.731364 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1051/2097] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:25:19.732639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240f850-38c1-40dc-819a-c46dc17f18f3 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:19.743340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240f850-38c1-40dc-819a-c46dc17f18f3 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:19.748313 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b6d4d-04fe-47f2-8a68-4cb809ad6583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:19.748984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b6d4d-04fe-47f2-8a68-4cb809ad6583 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:19.749268 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b6d4d-04fe-47f2-8a68-4cb809ad6583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:19.749574 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b6d4d-04fe-47f2-8a68-4cb809ad6583 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:19.750271 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b6d4d-04fe-47f2-8a68-4cb809ad6583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:19.752164 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b6d4d-04fe-47f2-8a68-4cb809ad6583 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:19.782253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1047/2098] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:19 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:19.786887 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d563391-a3c6-4d00-90fc-1ff1c4747194 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:19.797996 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d563391-a3c6-4d00-90fc-1ff1c4747194 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:19.888415 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c3ff9-d0b5-49a6-8283-2d5559a6335a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:19.889596 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c3ff9-d0b5-49a6-8283-2d5559a6335a admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:19.890074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c3ff9-d0b5-49a6-8283-2d5559a6335a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:19.890451 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c3ff9-d0b5-49a6-8283-2d5559a6335a admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:19.892851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c3ff9-d0b5-49a6-8283-2d5559a6335a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:19.895378 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c3ff9-d0b5-49a6-8283-2d5559a6335a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:19.921460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1052/2099] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:19 2022] POST /identity/v3/users => generated 287 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:19.925796 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b6d4d-04fe-47f2-8a68-4cb809ad6583 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:19.939113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b6d4d-04fe-47f2-8a68-4cb809ad6583 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:19.946858 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f58ce-a88b-4a2e-8e2a-97c22a834cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:19.947770 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f58ce-a88b-4a2e-8e2a-97c22a834cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:19.948054 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f58ce-a88b-4a2e-8e2a-97c22a834cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:19.948362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f58ce-a88b-4a2e-8e2a-97c22a834cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:19.949128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f58ce-a88b-4a2e-8e2a-97c22a834cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:19.950710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f58ce-a88b-4a2e-8e2a-97c22a834cf2 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:19.964347 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1048/2100] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:19.968371 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c3ff9-d0b5-49a6-8283-2d5559a6335a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:19.978927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c3ff9-d0b5-49a6-8283-2d5559a6335a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:19.984029 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27049409-16d0-4753-81e2-5ea07351371e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:19.984735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27049409-16d0-4753-81e2-5ea07351371e admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:19.985017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27049409-16d0-4753-81e2-5ea07351371e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:19.985286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27049409-16d0-4753-81e2-5ea07351371e admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:19.986027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27049409-16d0-4753-81e2-5ea07351371e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:19.987641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27049409-16d0-4753-81e2-5ea07351371e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:20.006168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1053/2101] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:25:20.007683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f58ce-a88b-4a2e-8e2a-97c22a834cf2 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:20.017478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f58ce-a88b-4a2e-8e2a-97c22a834cf2 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:20.027514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29b673-afcf-49d4-bfc4-16d03b70f0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:20.028271 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29b673-afcf-49d4-bfc4-16d03b70f0e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:20.028601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29b673-afcf-49d4-bfc4-16d03b70f0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:20.028879 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29b673-afcf-49d4-bfc4-16d03b70f0e6 admin admin] PATH_INFO: `/v3/projects/1bf20f7d2ff04565a0c06ebf8de59e38/users/5941d9e1753f49b4ad16997e7fa4140f/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:20.050389 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29b673-afcf-49d4-bfc4-16d03b70f0e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf20f7d2ff04565a0c06ebf8de59e38, user_id=5941d9e1753f49b4ad16997e7fa4140f, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:20.053907 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29b673-afcf-49d4-bfc4-16d03b70f0e6 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:20.070592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1049/2102] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:20 2022] PUT /identity/v3/projects/1bf20f7d2ff04565a0c06ebf8de59e38/users/5941d9e1753f49b4ad16997e7fa4140f/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:20.080936 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee32f7-cbf7-4fad-bd62-c9184077dd35 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:20.081382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee32f7-cbf7-4fad-bd62-c9184077dd35 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:20.081692 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee32f7-cbf7-4fad-bd62-c9184077dd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:20.111459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-cdee32f7-cbf7-4fad-bd62-c9184077dd35 None None] MFA Rules not processed for user `5941d9e1753f49b4ad16997e7fa4140f`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:20.112661 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cdee32f7-cbf7-4fad-bd62-c9184077dd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:20.192629 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1054/2103] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:20.204513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29b673-afcf-49d4-bfc4-16d03b70f0e6 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:20.206434 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-3b29b673-afcf-49d4-bfc4-16d03b70f0e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:20.229017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:20.301174 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29b673-afcf-49d4-bfc4-16d03b70f0e6 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:20.307389 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de80627-c1f1-49db-acb0-0288461cd1dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:20.308047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de80627-c1f1-49db-acb0-0288461cd1dd service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:20.308434 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de80627-c1f1-49db-acb0-0288461cd1dd service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:20.308720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de80627-c1f1-49db-acb0-0288461cd1dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:20.317163 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de80627-c1f1-49db-acb0-0288461cd1dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:20.319391 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de80627-c1f1-49db-acb0-0288461cd1dd service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:20.344562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1050/2104] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:20.606079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee32f7-cbf7-4fad-bd62-c9184077dd35 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:20.654232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:20.679018 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee32f7-cbf7-4fad-bd62-c9184077dd35 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:20.684643 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc4f3c-1ce9-4e22-8357-2de89bf11b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:20.685520 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc4f3c-1ce9-4e22-8357-2de89bf11b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:20.685803 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc4f3c-1ce9-4e22-8357-2de89bf11b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:20.686079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc4f3c-1ce9-4e22-8357-2de89bf11b55 admin admin] PATH_INFO: `/v3/users/5941d9e1753f49b4ad16997e7fa4140f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:20.687536 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc4f3c-1ce9-4e22-8357-2de89bf11b55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5941d9e1753f49b4ad16997e7fa4140f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:20.689805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc4f3c-1ce9-4e22-8357-2de89bf11b55 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:20.737043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a3fc4f3c-1ce9-4e22-8357-2de89bf11b55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5941d9e1753f49b4ad16997e7fa4140f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:20.745922 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a3fc4f3c-1ce9-4e22-8357-2de89bf11b55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5941d9e1753f49b4ad16997e7fa4140f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:20.746202 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a3fc4f3c-1ce9-4e22-8357-2de89bf11b55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5941d9e1753f49b4ad16997e7fa4140f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:20.753830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a3fc4f3c-1ce9-4e22-8357-2de89bf11b55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5941d9e1753f49b4ad16997e7fa4140f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:20.766065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a3fc4f3c-1ce9-4e22-8357-2de89bf11b55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5941d9e1753f49b4ad16997e7fa4140f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:20.773113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1055/2105] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:20 2022] DELETE /identity/v3/users/5941d9e1753f49b4ad16997e7fa4140f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:20.777677 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e114a-941c-4e78-b121-9d624939109a None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:20.777973 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e114a-941c-4e78-b121-9d624939109a None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:20.778254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e114a-941c-4e78-b121-9d624939109a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:20.795656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-be5e114a-941c-4e78-b121-9d624939109a None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:20.796915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-be5e114a-941c-4e78-b121-9d624939109a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:20.857805 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1051/2106] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:20.867658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc4f3c-1ce9-4e22-8357-2de89bf11b55 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:20.869340 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a3fc4f3c-1ce9-4e22-8357-2de89bf11b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:20.927338 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc4f3c-1ce9-4e22-8357-2de89bf11b55 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:20.930954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555c84a-bd32-4875-83c2-4c4bd0901b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:20.931598 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555c84a-bd32-4875-83c2-4c4bd0901b78 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:20.931883 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555c84a-bd32-4875-83c2-4c4bd0901b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:20.932160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555c84a-bd32-4875-83c2-4c4bd0901b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:20.943323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555c84a-bd32-4875-83c2-4c4bd0901b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:20.944828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555c84a-bd32-4875-83c2-4c4bd0901b78 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:20.954725 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1056/2107] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:21.322192 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e114a-941c-4e78-b121-9d624939109a None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:21.323636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-be5e114a-941c-4e78-b121-9d624939109a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:21.338197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e114a-941c-4e78-b121-9d624939109a None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:21.343517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed5222-7740-47e2-854d-14e8776db52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:21.344440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed5222-7740-47e2-854d-14e8776db52f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:21.344722 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed5222-7740-47e2-854d-14e8776db52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:21.344995 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed5222-7740-47e2-854d-14e8776db52f admin admin] PATH_INFO: `/v3/projects/1bf20f7d2ff04565a0c06ebf8de59e38` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:21.346279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed5222-7740-47e2-854d-14e8776db52f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bf20f7d2ff04565a0c06ebf8de59e38)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:21.348216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed5222-7740-47e2-854d-14e8776db52f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:21.401668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5eed5222-7740-47e2-854d-14e8776db52f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bf20f7d2ff04565a0c06ebf8de59e38', 'request_id': 'req-5eed5222-7740-47e2-854d-14e8776db52f'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:21.411335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5eed5222-7740-47e2-854d-14e8776db52f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bf20f7d2ff04565a0c06ebf8de59e38', 'request_id': 'req-5eed5222-7740-47e2-854d-14e8776db52f'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:21.421513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5eed5222-7740-47e2-854d-14e8776db52f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bf20f7d2ff04565a0c06ebf8de59e38', 'request_id': 'req-5eed5222-7740-47e2-854d-14e8776db52f'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:21.435991 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5eed5222-7740-47e2-854d-14e8776db52f admin admin] The token cache is being invalidate because project 1bf20f7d2ff04565a0c06ebf8de59e38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:21.436240 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5eed5222-7740-47e2-854d-14e8776db52f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:21.437375 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5eed5222-7740-47e2-854d-14e8776db52f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:21.446699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1052/2108] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:21 2022] DELETE /identity/v3/projects/1bf20f7d2ff04565a0c06ebf8de59e38 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:21.451721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea628813-a54b-4a8f-9d14-040dc143aa25 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:21.452145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea628813-a54b-4a8f-9d14-040dc143aa25 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:21.452893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea628813-a54b-4a8f-9d14-040dc143aa25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:21.471145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ea628813-a54b-4a8f-9d14-040dc143aa25 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:21.472633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ea628813-a54b-4a8f-9d14-040dc143aa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:21.547274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1057/2109] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:21.556474 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df271b-5c09-490e-89aa-d1a3678de491 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:21.557166 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df271b-5c09-490e-89aa-d1a3678de491 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:21.557618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df271b-5c09-490e-89aa-d1a3678de491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:21.583802 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-61df271b-5c09-490e-89aa-d1a3678de491 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:21.586248 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-61df271b-5c09-490e-89aa-d1a3678de491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:21.594367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1053/2110] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:21.598819 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea628813-a54b-4a8f-9d14-040dc143aa25 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:21.604105 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:21.610153 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea628813-a54b-4a8f-9d14-040dc143aa25 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:21.618112 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2f027-6e3e-4389-bdae-2f0ea7ad4f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:21.619587 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2f027-6e3e-4389-bdae-2f0ea7ad4f2a admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:21.620230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2f027-6e3e-4389-bdae-2f0ea7ad4f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:21.620798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2f027-6e3e-4389-bdae-2f0ea7ad4f2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:21.622644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2f027-6e3e-4389-bdae-2f0ea7ad4f2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:21.623379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2f027-6e3e-4389-bdae-2f0ea7ad4f2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:21.625975 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2f027-6e3e-4389-bdae-2f0ea7ad4f2a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:21.640019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1058/2111] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:25:21.644821 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df271b-5c09-490e-89aa-d1a3678de491 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:21.654639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df271b-5c09-490e-89aa-d1a3678de491 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:21.660533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8dcfa-e14e-48d2-9b39-648e2e6a4acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:21.662086 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8dcfa-e14e-48d2-9b39-648e2e6a4acc admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:21.662515 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8dcfa-e14e-48d2-9b39-648e2e6a4acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:21.662865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8dcfa-e14e-48d2-9b39-648e2e6a4acc admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:21.663871 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8dcfa-e14e-48d2-9b39-648e2e6a4acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:21.666187 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8dcfa-e14e-48d2-9b39-648e2e6a4acc admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:21.678790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:21.700901 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1054/2112] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:21 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:21.702081 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2f027-6e3e-4389-bdae-2f0ea7ad4f2a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:21.718903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2f027-6e3e-4389-bdae-2f0ea7ad4f2a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:21.790134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4dd34-c055-46ac-a1ef-db9528ffba0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:21.790899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4dd34-c055-46ac-a1ef-db9528ffba0b admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:21.791270 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4dd34-c055-46ac-a1ef-db9528ffba0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:21.791965 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4dd34-c055-46ac-a1ef-db9528ffba0b admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:21.792850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4dd34-c055-46ac-a1ef-db9528ffba0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:21.795009 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4dd34-c055-46ac-a1ef-db9528ffba0b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:21.819395 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1059/2113] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:21 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:21.824007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8dcfa-e14e-48d2-9b39-648e2e6a4acc admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:21.836968 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8dcfa-e14e-48d2-9b39-648e2e6a4acc admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:21.844077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b28b0-6c7a-44cf-a57b-8315998da9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:21.844800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b28b0-6c7a-44cf-a57b-8315998da9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:21.845278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b28b0-6c7a-44cf-a57b-8315998da9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:21.845564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b28b0-6c7a-44cf-a57b-8315998da9d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:21.846309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b28b0-6c7a-44cf-a57b-8315998da9d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:21.848091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b28b0-6c7a-44cf-a57b-8315998da9d5 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:21.865474 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1055/2114] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:21.866386 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4dd34-c055-46ac-a1ef-db9528ffba0b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:21.878934 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4dd34-c055-46ac-a1ef-db9528ffba0b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:21.885283 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6effb8-9cd5-4607-9047-e56fdbaf8801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:21.885960 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6effb8-9cd5-4607-9047-e56fdbaf8801 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:21.886252 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6effb8-9cd5-4607-9047-e56fdbaf8801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:21.886661 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6effb8-9cd5-4607-9047-e56fdbaf8801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:21.887281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6effb8-9cd5-4607-9047-e56fdbaf8801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:21.889116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6effb8-9cd5-4607-9047-e56fdbaf8801 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:21.904472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1060/2115] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:21.906146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b28b0-6c7a-44cf-a57b-8315998da9d5 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:21.917016 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b28b0-6c7a-44cf-a57b-8315998da9d5 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:21.921496 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf20ac5-d197-4067-8ed7-8291573064d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:21.922374 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf20ac5-d197-4067-8ed7-8291573064d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:21.922723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf20ac5-d197-4067-8ed7-8291573064d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:21.923003 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf20ac5-d197-4067-8ed7-8291573064d2 admin admin] PATH_INFO: `/v3/projects/4beb658c668d46bfb81edadc52d8cd18/users/125c737c8f8840409f0f81cbf12b636a/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:21.940366 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf20ac5-d197-4067-8ed7-8291573064d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4beb658c668d46bfb81edadc52d8cd18, user_id=125c737c8f8840409f0f81cbf12b636a, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:21.943596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf20ac5-d197-4067-8ed7-8291573064d2 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:21.958001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1056/2116] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:21 2022] PUT /identity/v3/projects/4beb658c668d46bfb81edadc52d8cd18/users/125c737c8f8840409f0f81cbf12b636a/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:21.970397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbc869-f434-415f-9931-3edb3456b93d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:21.970689 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbc869-f434-415f-9931-3edb3456b93d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:21.970972 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbc869-f434-415f-9931-3edb3456b93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:21.999921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5efbc869-f434-415f-9931-3edb3456b93d None None] MFA Rules not processed for user `125c737c8f8840409f0f81cbf12b636a`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:22.001169 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5efbc869-f434-415f-9931-3edb3456b93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:22.067528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1061/2117] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:22.072253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf20ac5-d197-4067-8ed7-8291573064d2 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:22.133975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf20ac5-d197-4067-8ed7-8291573064d2 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:22.139754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac64a5-94cd-4e0f-8a98-ff1f22475803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:22.140513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac64a5-94cd-4e0f-8a98-ff1f22475803 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:22.140800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac64a5-94cd-4e0f-8a98-ff1f22475803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:22.141080 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac64a5-94cd-4e0f-8a98-ff1f22475803 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:22.141781 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac64a5-94cd-4e0f-8a98-ff1f22475803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:22.144321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac64a5-94cd-4e0f-8a98-ff1f22475803 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:22.171609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1057/2118] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:22 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:22.175908 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbc869-f434-415f-9931-3edb3456b93d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:22.186877 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbc869-f434-415f-9931-3edb3456b93d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:22.237138 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:22.244525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841732f-6190-415b-9bea-51b112ad1914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:22.245365 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841732f-6190-415b-9bea-51b112ad1914 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:22.245814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841732f-6190-415b-9bea-51b112ad1914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:22.246315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841732f-6190-415b-9bea-51b112ad1914 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:22.247049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b841732f-6190-415b-9bea-51b112ad1914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:22.249077 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b841732f-6190-415b-9bea-51b112ad1914 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:22.270344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1062/2119] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:22 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:25:22.272682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac64a5-94cd-4e0f-8a98-ff1f22475803 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:22.284354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac64a5-94cd-4e0f-8a98-ff1f22475803 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:22.289534 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace63d8-4fdf-4985-aae0-c5ce320369d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:22.290212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace63d8-4fdf-4985-aae0-c5ce320369d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:22.290491 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace63d8-4fdf-4985-aae0-c5ce320369d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:22.290764 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace63d8-4fdf-4985-aae0-c5ce320369d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:22.291494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace63d8-4fdf-4985-aae0-c5ce320369d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:22.293126 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace63d8-4fdf-4985-aae0-c5ce320369d8 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:22.307355 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1058/2120] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:22.308899 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841732f-6190-415b-9bea-51b112ad1914 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:22.320558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841732f-6190-415b-9bea-51b112ad1914 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:22.325827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ec429-8677-42fb-b301-1c215777b0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:22.326717 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ec429-8677-42fb-b301-1c215777b0bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:22.327515 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ec429-8677-42fb-b301-1c215777b0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:22.327515 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ec429-8677-42fb-b301-1c215777b0bb admin admin] PATH_INFO: `/v3/projects/ab6a9224a00b43e082aea2ca7267313d/users/3e1014862df34aca9c4fb047c1806fcc/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:22.346893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ec429-8677-42fb-b301-1c215777b0bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab6a9224a00b43e082aea2ca7267313d, user_id=3e1014862df34aca9c4fb047c1806fcc, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:22.351987 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ec429-8677-42fb-b301-1c215777b0bb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:22.369383 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1063/2121] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:22 2022] PUT /identity/v3/projects/ab6a9224a00b43e082aea2ca7267313d/users/3e1014862df34aca9c4fb047c1806fcc/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:22.373128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace63d8-4fdf-4985-aae0-c5ce320369d8 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:22.389346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace63d8-4fdf-4985-aae0-c5ce320369d8 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:22.448127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbae9d-e9ee-42da-8a91-2ff2063972d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:22.448946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbae9d-e9ee-42da-8a91-2ff2063972d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:22.449300 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbae9d-e9ee-42da-8a91-2ff2063972d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:22.449763 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbae9d-e9ee-42da-8a91-2ff2063972d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:22.450455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbae9d-e9ee-42da-8a91-2ff2063972d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:22.452335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbae9d-e9ee-42da-8a91-2ff2063972d6 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:22.465070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1059/2122] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:22.469106 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ec429-8677-42fb-b301-1c215777b0bb admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:22.482610 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ec429-8677-42fb-b301-1c215777b0bb admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:22.488232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623906c-5082-4eb8-b67c-ffff6c6ef4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:22.488698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623906c-5082-4eb8-b67c-ffff6c6ef4a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:22.488984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623906c-5082-4eb8-b67c-ffff6c6ef4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:22.489265 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623906c-5082-4eb8-b67c-ffff6c6ef4a2 admin admin] PATH_INFO: `/v3/projects/ab6a9224a00b43e082aea2ca7267313d/users/3e1014862df34aca9c4fb047c1806fcc/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:22.491339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623906c-5082-4eb8-b67c-ffff6c6ef4a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab6a9224a00b43e082aea2ca7267313d, user_id=3e1014862df34aca9c4fb047c1806fcc, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:22.494141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623906c-5082-4eb8-b67c-ffff6c6ef4a2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:22.504827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1064/2123] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:22 2022] PUT /identity/v3/projects/ab6a9224a00b43e082aea2ca7267313d/users/3e1014862df34aca9c4fb047c1806fcc/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:22.515655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb4dab-a7bd-408e-b38b-fe1389c70539 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:22.515949 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb4dab-a7bd-408e-b38b-fe1389c70539 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:22.516229 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb4dab-a7bd-408e-b38b-fe1389c70539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:22.543803 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-9cbb4dab-a7bd-408e-b38b-fe1389c70539 None None] MFA Rules not processed for user `3e1014862df34aca9c4fb047c1806fcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:22.545070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-9cbb4dab-a7bd-408e-b38b-fe1389c70539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:22.558546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:22.616156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1060/2124] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:22.627326 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623906c-5082-4eb8-b67c-ffff6c6ef4a2 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:22.629608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5623906c-5082-4eb8-b67c-ffff6c6ef4a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:22.662941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb4dab-a7bd-408e-b38b-fe1389c70539 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:22.664992 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-9cbb4dab-a7bd-408e-b38b-fe1389c70539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:22.711822 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623906c-5082-4eb8-b67c-ffff6c6ef4a2 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:22.715852 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c933eb-b556-4385-8dc1-f340928e843d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:22.716533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c933eb-b556-4385-8dc1-f340928e843d service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:22.716822 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c933eb-b556-4385-8dc1-f340928e843d service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:22.717097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c933eb-b556-4385-8dc1-f340928e843d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:22.727412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c933eb-b556-4385-8dc1-f340928e843d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:22.728949 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c933eb-b556-4385-8dc1-f340928e843d service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:22.740661 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1065/2125] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:22.757942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb4dab-a7bd-408e-b38b-fe1389c70539 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:22.769998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2b368-7d1c-42aa-b898-f5ad00841bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:22.769998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2b368-7d1c-42aa-b898-f5ad00841bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:22.769998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2b368-7d1c-42aa-b898-f5ad00841bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:22.769998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2b368-7d1c-42aa-b898-f5ad00841bb6 admin admin] PATH_INFO: `/v3/users/9e45c027745646fc865f0ee5e44266f1` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:22.770341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a2b368-7d1c-42aa-b898-f5ad00841bb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e45c027745646fc865f0ee5e44266f1)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:22.770341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a2b368-7d1c-42aa-b898-f5ad00841bb6 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:22.771326 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c933eb-b556-4385-8dc1-f340928e843d service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:22.782210 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c933eb-b556-4385-8dc1-f340928e843d service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:22.787040 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081c125-a70b-4469-9671-01cb95322315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:22.787921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081c125-a70b-4469-9671-01cb95322315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:22.788220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081c125-a70b-4469-9671-01cb95322315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:22.788538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081c125-a70b-4469-9671-01cb95322315 admin admin] PATH_INFO: `/v3/users/125c737c8f8840409f0f81cbf12b636a` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:22.789787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081c125-a70b-4469-9671-01cb95322315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=125c737c8f8840409f0f81cbf12b636a)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:22.791718 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081c125-a70b-4469-9671-01cb95322315 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:22.829317 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e9a2b368-7d1c-42aa-b898-f5ad00841bb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e45c027745646fc865f0ee5e44266f1'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:22.838406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e9a2b368-7d1c-42aa-b898-f5ad00841bb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e45c027745646fc865f0ee5e44266f1'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:22.854208 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4081c125-a70b-4469-9671-01cb95322315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '125c737c8f8840409f0f81cbf12b636a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:22.854545 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e9a2b368-7d1c-42aa-b898-f5ad00841bb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e45c027745646fc865f0ee5e44266f1'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:22.863237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4081c125-a70b-4469-9671-01cb95322315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '125c737c8f8840409f0f81cbf12b636a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:22.863918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4081c125-a70b-4469-9671-01cb95322315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '125c737c8f8840409f0f81cbf12b636a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:22.865553 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e9a2b368-7d1c-42aa-b898-f5ad00841bb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e45c027745646fc865f0ee5e44266f1'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:22.866414 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e9a2b368-7d1c-42aa-b898-f5ad00841bb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e45c027745646fc865f0ee5e44266f1'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:22.872895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4081c125-a70b-4469-9671-01cb95322315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '125c737c8f8840409f0f81cbf12b636a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:22.876922 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1061/2126] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:22 2022] DELETE /identity/v3/users/9e45c027745646fc865f0ee5e44266f1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:22.883022 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94eda8-2c62-4c5f-94ef-2cf4071f092a None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:22.883398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94eda8-2c62-4c5f-94ef-2cf4071f092a None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:22.884002 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94eda8-2c62-4c5f-94ef-2cf4071f092a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:22.887127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4081c125-a70b-4469-9671-01cb95322315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '125c737c8f8840409f0f81cbf12b636a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:22.894100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1066/2127] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:22 2022] DELETE /identity/v3/users/125c737c8f8840409f0f81cbf12b636a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:22.898577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d96ee-2575-47ff-a562-678ce4b95ecd None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:22.898939 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d96ee-2575-47ff-a562-678ce4b95ecd None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:22.899397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d96ee-2575-47ff-a562-678ce4b95ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:22.905511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-0d94eda8-2c62-4c5f-94ef-2cf4071f092a None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:22.906758 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0d94eda8-2c62-4c5f-94ef-2cf4071f092a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:22.919833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-6a6d96ee-2575-47ff-a562-678ce4b95ecd None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:22.921210 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6a6d96ee-2575-47ff-a562-678ce4b95ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:22.939597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:22.953077 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:22.984746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1062/2128] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:22.995704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94eda8-2c62-4c5f-94ef-2cf4071f092a None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:22.996994 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0d94eda8-2c62-4c5f-94ef-2cf4071f092a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:23.001801 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1067/2129] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:23.014792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d96ee-2575-47ff-a562-678ce4b95ecd None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:23.016981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6a6d96ee-2575-47ff-a562-678ce4b95ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:23.085234 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d96ee-2575-47ff-a562-678ce4b95ecd None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:23.089082 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592c46a-df43-4e69-87b4-eeaedc93ef65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:23.089732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592c46a-df43-4e69-87b4-eeaedc93ef65 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:23.089939 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94eda8-2c62-4c5f-94ef-2cf4071f092a None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:23.090059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592c46a-df43-4e69-87b4-eeaedc93ef65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:23.090368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592c46a-df43-4e69-87b4-eeaedc93ef65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:23.094439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e00b786-edcf-444e-a4d1-2bbad0daacd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:23.095148 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00b786-edcf-444e-a4d1-2bbad0daacd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:23.095433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00b786-edcf-444e-a4d1-2bbad0daacd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:23.095843 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00b786-edcf-444e-a4d1-2bbad0daacd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:23.107053 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e00b786-edcf-444e-a4d1-2bbad0daacd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:23.108516 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e00b786-edcf-444e-a4d1-2bbad0daacd8 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:23.110356 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f592c46a-df43-4e69-87b4-eeaedc93ef65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:23.111827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f592c46a-df43-4e69-87b4-eeaedc93ef65 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:23.118244 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1063/2130] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:25:23.124166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1068/2131] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:25:23.681415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e00b786-edcf-444e-a4d1-2bbad0daacd8 service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:23.682869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-3e00b786-edcf-444e-a4d1-2bbad0daacd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:23.688411 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592c46a-df43-4e69-87b4-eeaedc93ef65 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:23.689989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f592c46a-df43-4e69-87b4-eeaedc93ef65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:23.698736 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e00b786-edcf-444e-a4d1-2bbad0daacd8 service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:23.704923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592c46a-df43-4e69-87b4-eeaedc93ef65 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:23.709972 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478ed7e-5398-42d5-a6fa-178c04b689f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:23.710670 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59e932-35b7-412c-b547-f13f6caad507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:23.710826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478ed7e-5398-42d5-a6fa-178c04b689f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:23.711121 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478ed7e-5398-42d5-a6fa-178c04b689f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:23.711398 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478ed7e-5398-42d5-a6fa-178c04b689f2 admin admin] PATH_INFO: `/v3/projects/a611e0777d8d43b58895bd584509c1f2` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:23.711551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59e932-35b7-412c-b547-f13f6caad507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:23.711817 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59e932-35b7-412c-b547-f13f6caad507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:23.712092 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59e932-35b7-412c-b547-f13f6caad507 admin admin] PATH_INFO: `/v3/projects/4beb658c668d46bfb81edadc52d8cd18` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:23.712711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478ed7e-5398-42d5-a6fa-178c04b689f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a611e0777d8d43b58895bd584509c1f2)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:23.713314 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59e932-35b7-412c-b547-f13f6caad507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4beb658c668d46bfb81edadc52d8cd18)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:23.714790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478ed7e-5398-42d5-a6fa-178c04b689f2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:23.715211 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59e932-35b7-412c-b547-f13f6caad507 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:23.770845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0478ed7e-5398-42d5-a6fa-178c04b689f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a611e0777d8d43b58895bd584509c1f2', 'request_id': 'req-0478ed7e-5398-42d5-a6fa-178c04b689f2'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:23.771699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5c59e932-35b7-412c-b547-f13f6caad507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4beb658c668d46bfb81edadc52d8cd18', 'request_id': 'req-5c59e932-35b7-412c-b547-f13f6caad507'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:23.777142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0478ed7e-5398-42d5-a6fa-178c04b689f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a611e0777d8d43b58895bd584509c1f2', 'request_id': 'req-0478ed7e-5398-42d5-a6fa-178c04b689f2'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:23.779297 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5c59e932-35b7-412c-b547-f13f6caad507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4beb658c668d46bfb81edadc52d8cd18', 'request_id': 'req-5c59e932-35b7-412c-b547-f13f6caad507'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:23.785028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0478ed7e-5398-42d5-a6fa-178c04b689f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a611e0777d8d43b58895bd584509c1f2', 'request_id': 'req-0478ed7e-5398-42d5-a6fa-178c04b689f2'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:23.785326 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5c59e932-35b7-412c-b547-f13f6caad507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4beb658c668d46bfb81edadc52d8cd18', 'request_id': 'req-5c59e932-35b7-412c-b547-f13f6caad507'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:23.796105 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0478ed7e-5398-42d5-a6fa-178c04b689f2 admin admin] The token cache is being invalidate because project a611e0777d8d43b58895bd584509c1f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:23.796424 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0478ed7e-5398-42d5-a6fa-178c04b689f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:23.797087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0478ed7e-5398-42d5-a6fa-178c04b689f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:23.797642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5c59e932-35b7-412c-b547-f13f6caad507 admin admin] The token cache is being invalidate because project 4beb658c668d46bfb81edadc52d8cd18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:23.798258 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5c59e932-35b7-412c-b547-f13f6caad507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:23.798637 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1069/2132] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:23 2022] DELETE /identity/v3/projects/a611e0777d8d43b58895bd584509c1f2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:23.800677 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5c59e932-35b7-412c-b547-f13f6caad507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:23.803508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1064/2133] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:23 2022] DELETE /identity/v3/projects/4beb658c668d46bfb81edadc52d8cd18 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:23.811658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478ed7e-5398-42d5-a6fa-178c04b689f2 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:23.813268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0478ed7e-5398-42d5-a6fa-178c04b689f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:23.874082 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:23.895705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478ed7e-5398-42d5-a6fa-178c04b689f2 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:23.959985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5162510-86e4-4fc6-8634-94770930e6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:23.960917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5162510-86e4-4fc6-8634-94770930e6bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:23.961206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5162510-86e4-4fc6-8634-94770930e6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:23.961487 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5162510-86e4-4fc6-8634-94770930e6bb admin admin] PATH_INFO: `/v3/users/3e1014862df34aca9c4fb047c1806fcc` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:23.962755 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5162510-86e4-4fc6-8634-94770930e6bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e1014862df34aca9c4fb047c1806fcc)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:23.964759 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5162510-86e4-4fc6-8634-94770930e6bb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:24.023042 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f5162510-86e4-4fc6-8634-94770930e6bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e1014862df34aca9c4fb047c1806fcc'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:24.031062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f5162510-86e4-4fc6-8634-94770930e6bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e1014862df34aca9c4fb047c1806fcc'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:24.031749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f5162510-86e4-4fc6-8634-94770930e6bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e1014862df34aca9c4fb047c1806fcc'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:24.039585 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f5162510-86e4-4fc6-8634-94770930e6bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e1014862df34aca9c4fb047c1806fcc'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:24.051596 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f5162510-86e4-4fc6-8634-94770930e6bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e1014862df34aca9c4fb047c1806fcc'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:24.058204 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1070/2134] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:23 2022] DELETE /identity/v3/users/3e1014862df34aca9c4fb047c1806fcc => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:24.390536 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59e932-35b7-412c-b547-f13f6caad507 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:24.392064 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5c59e932-35b7-412c-b547-f13f6caad507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:24.406420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59e932-35b7-412c-b547-f13f6caad507 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:24.461989 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:24.469246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007238d-7825-49af-a819-64711ff800f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:24.470504 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007238d-7825-49af-a819-64711ff800f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:24.470925 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007238d-7825-49af-a819-64711ff800f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:24.471214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007238d-7825-49af-a819-64711ff800f6 admin admin] PATH_INFO: `/v3/projects/ab6a9224a00b43e082aea2ca7267313d` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:24.472527 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007238d-7825-49af-a819-64711ff800f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab6a9224a00b43e082aea2ca7267313d)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:24.475506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007238d-7825-49af-a819-64711ff800f6 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:24.526395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a007238d-7825-49af-a819-64711ff800f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab6a9224a00b43e082aea2ca7267313d', 'request_id': 'req-a007238d-7825-49af-a819-64711ff800f6'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:24.535398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a007238d-7825-49af-a819-64711ff800f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab6a9224a00b43e082aea2ca7267313d', 'request_id': 'req-a007238d-7825-49af-a819-64711ff800f6'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:24.542076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a007238d-7825-49af-a819-64711ff800f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab6a9224a00b43e082aea2ca7267313d', 'request_id': 'req-a007238d-7825-49af-a819-64711ff800f6'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:24.553168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a007238d-7825-49af-a819-64711ff800f6 admin admin] The token cache is being invalidate because project ab6a9224a00b43e082aea2ca7267313d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:24.553461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a007238d-7825-49af-a819-64711ff800f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:24.554122 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a007238d-7825-49af-a819-64711ff800f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:24.555680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1065/2135] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:24 2022] DELETE /identity/v3/projects/ab6a9224a00b43e082aea2ca7267313d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:24.560614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23bee3-bd29-4705-ba19-4291508d831e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:24.560896 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23bee3-bd29-4705-ba19-4291508d831e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:24.561181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23bee3-bd29-4705-ba19-4291508d831e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:24.581046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7e23bee3-bd29-4705-ba19-4291508d831e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:24.582265 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7e23bee3-bd29-4705-ba19-4291508d831e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:24.651087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1071/2136] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:24.658818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdaec5-c0c9-4084-a280-b204ccb84846 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:24.659138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdaec5-c0c9-4084-a280-b204ccb84846 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:24.659416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdaec5-c0c9-4084-a280-b204ccb84846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:24.681479 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-09bdaec5-c0c9-4084-a280-b204ccb84846 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:24.682572 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-09bdaec5-c0c9-4084-a280-b204ccb84846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:24.695608 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1066/2137] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:24.700418 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23bee3-bd29-4705-ba19-4291508d831e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:24.711419 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23bee3-bd29-4705-ba19-4291508d831e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:24.718645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff893755-09de-478b-b7b4-13c2811a2deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:24.718645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff893755-09de-478b-b7b4-13c2811a2deb admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:24.718645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff893755-09de-478b-b7b4-13c2811a2deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:24.718645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff893755-09de-478b-b7b4-13c2811a2deb admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:24.719811 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff893755-09de-478b-b7b4-13c2811a2deb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:24.720718 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff893755-09de-478b-b7b4-13c2811a2deb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:24.722282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff893755-09de-478b-b7b4-13c2811a2deb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:24.744242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1072/2138] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:25:24.744506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdaec5-c0c9-4084-a280-b204ccb84846 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:24.761118 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdaec5-c0c9-4084-a280-b204ccb84846 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:24.766387 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cb486-0760-4bb0-b6ce-f792c6dbadd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:24.767331 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cb486-0760-4bb0-b6ce-f792c6dbadd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:24.767568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cb486-0760-4bb0-b6ce-f792c6dbadd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:24.767856 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cb486-0760-4bb0-b6ce-f792c6dbadd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:24.770819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cb486-0760-4bb0-b6ce-f792c6dbadd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:24.772792 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cb486-0760-4bb0-b6ce-f792c6dbadd0 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:24.806100 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1067/2139] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:24 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:25:24.810389 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff893755-09de-478b-b7b4-13c2811a2deb admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:24.818095 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:24.828932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff893755-09de-478b-b7b4-13c2811a2deb admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:24.911062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156aece9-5a35-4dbb-a1e8-5e4d14a546e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:24.912266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aece9-5a35-4dbb-a1e8-5e4d14a546e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:24.912636 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aece9-5a35-4dbb-a1e8-5e4d14a546e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:24.913529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aece9-5a35-4dbb-a1e8-5e4d14a546e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:24.913616 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156aece9-5a35-4dbb-a1e8-5e4d14a546e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:24.915880 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156aece9-5a35-4dbb-a1e8-5e4d14a546e7 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:24.941432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1073/2140] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:24 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:24.942892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cb486-0760-4bb0-b6ce-f792c6dbadd0 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:24.957297 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cb486-0760-4bb0-b6ce-f792c6dbadd0 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:24.962513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c28c6-9147-4911-b0e4-9b366bcc1365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:24.963203 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c28c6-9147-4911-b0e4-9b366bcc1365 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:24.963486 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c28c6-9147-4911-b0e4-9b366bcc1365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:24.963761 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c28c6-9147-4911-b0e4-9b366bcc1365 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:24.964523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c28c6-9147-4911-b0e4-9b366bcc1365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:24.966091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c28c6-9147-4911-b0e4-9b366bcc1365 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:24.980329 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1068/2141] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:24.983141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156aece9-5a35-4dbb-a1e8-5e4d14a546e7 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:24.993535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156aece9-5a35-4dbb-a1e8-5e4d14a546e7 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:24.998830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597f8a2-20ff-4859-b8ed-1075f26f5ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:24.999493 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597f8a2-20ff-4859-b8ed-1075f26f5ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:24.999778 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597f8a2-20ff-4859-b8ed-1075f26f5ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:25.000047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597f8a2-20ff-4859-b8ed-1075f26f5ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:25.000829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597f8a2-20ff-4859-b8ed-1075f26f5ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:25.002442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597f8a2-20ff-4859-b8ed-1075f26f5ad2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:25.016250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1074/2142] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:25.018152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c28c6-9147-4911-b0e4-9b366bcc1365 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:25.029210 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c28c6-9147-4911-b0e4-9b366bcc1365 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:25.033687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336885eb-26d1-4309-83e4-1e34fa968561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:25.034637 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336885eb-26d1-4309-83e4-1e34fa968561 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:25.034844 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336885eb-26d1-4309-83e4-1e34fa968561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:25.035117 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336885eb-26d1-4309-83e4-1e34fa968561 admin admin] PATH_INFO: `/v3/projects/cd00901619844968a5fe11a39636669b/users/ca03ddc164b94195b40777d0eba0cc11/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:25.053200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336885eb-26d1-4309-83e4-1e34fa968561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd00901619844968a5fe11a39636669b, user_id=ca03ddc164b94195b40777d0eba0cc11, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:25.055841 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336885eb-26d1-4309-83e4-1e34fa968561 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:25.071355 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1069/2143] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:25 2022] PUT /identity/v3/projects/cd00901619844968a5fe11a39636669b/users/ca03ddc164b94195b40777d0eba0cc11/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:25.083898 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2bc1d-7d8e-41aa-974e-b4cc2225c392 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:25.084190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2bc1d-7d8e-41aa-974e-b4cc2225c392 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:25.084502 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2bc1d-7d8e-41aa-974e-b4cc2225c392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:25.115610 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c6d2bc1d-7d8e-41aa-974e-b4cc2225c392 None None] MFA Rules not processed for user `ca03ddc164b94195b40777d0eba0cc11`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:25.116887 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c6d2bc1d-7d8e-41aa-974e-b4cc2225c392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:25.189515 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1075/2144] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:25.194352 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336885eb-26d1-4309-83e4-1e34fa968561 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:25.258413 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336885eb-26d1-4309-83e4-1e34fa968561 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:25.263108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bb257-cd18-4e39-9219-e408e33f9b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:25.263807 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bb257-cd18-4e39-9219-e408e33f9b2b admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:25.264203 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bb257-cd18-4e39-9219-e408e33f9b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:25.264597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bb257-cd18-4e39-9219-e408e33f9b2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:25.265382 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bb257-cd18-4e39-9219-e408e33f9b2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:25.267446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bb257-cd18-4e39-9219-e408e33f9b2b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:25.294338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1070/2145] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:25 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:25.298281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2bc1d-7d8e-41aa-974e-b4cc2225c392 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:25.309011 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2bc1d-7d8e-41aa-974e-b4cc2225c392 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:25.369583 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bed1d0-8f48-484d-80f8-867de0738479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:25.370265 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bed1d0-8f48-484d-80f8-867de0738479 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:25.370552 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bed1d0-8f48-484d-80f8-867de0738479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:25.370836 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bed1d0-8f48-484d-80f8-867de0738479 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:25.371531 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bed1d0-8f48-484d-80f8-867de0738479 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:25.373484 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bed1d0-8f48-484d-80f8-867de0738479 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:25.395114 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1076/2146] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:25 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:25.399689 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bb257-cd18-4e39-9219-e408e33f9b2b admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:25.410057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bb257-cd18-4e39-9219-e408e33f9b2b admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:25.416282 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ee9b8-284b-4fbf-8b32-3fcaadec5569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:25.416986 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ee9b8-284b-4fbf-8b32-3fcaadec5569 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:25.417304 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ee9b8-284b-4fbf-8b32-3fcaadec5569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:25.417534 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ee9b8-284b-4fbf-8b32-3fcaadec5569 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:25.418861 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ee9b8-284b-4fbf-8b32-3fcaadec5569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:25.419856 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ee9b8-284b-4fbf-8b32-3fcaadec5569 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:25.422402 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:25.431966 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1071/2147] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:25.436408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bed1d0-8f48-484d-80f8-867de0738479 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:25.442440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:25.449982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bed1d0-8f48-484d-80f8-867de0738479 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:25.455189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e6003-a679-42e9-b9f2-e95721fd0271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:25.456143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e6003-a679-42e9-b9f2-e95721fd0271 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:25.456470 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e6003-a679-42e9-b9f2-e95721fd0271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:25.456752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e6003-a679-42e9-b9f2-e95721fd0271 admin admin] PATH_INFO: `/v3/projects/0826357a0e2c46a8bce26a1b0c49bae9/users/366b67832dd74a08855add2f54344f51/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:25.477062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e6003-a679-42e9-b9f2-e95721fd0271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0826357a0e2c46a8bce26a1b0c49bae9, user_id=366b67832dd74a08855add2f54344f51, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:25.480109 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e6003-a679-42e9-b9f2-e95721fd0271 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:25.495341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1077/2148] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:25 2022] PUT /identity/v3/projects/0826357a0e2c46a8bce26a1b0c49bae9/users/366b67832dd74a08855add2f54344f51/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:25.499577 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ee9b8-284b-4fbf-8b32-3fcaadec5569 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:25.516782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ee9b8-284b-4fbf-8b32-3fcaadec5569 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:25.571093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c1130-86d8-44ba-a26c-c54397bc9d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:25.571860 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c1130-86d8-44ba-a26c-c54397bc9d9f admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:25.572144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c1130-86d8-44ba-a26c-c54397bc9d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:25.572444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c1130-86d8-44ba-a26c-c54397bc9d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:25.573179 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c1130-86d8-44ba-a26c-c54397bc9d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:25.574763 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c1130-86d8-44ba-a26c-c54397bc9d9f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:25.589157 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1072/2149] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:25.590580 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e6003-a679-42e9-b9f2-e95721fd0271 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:25.601555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e6003-a679-42e9-b9f2-e95721fd0271 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:25.606197 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffedd6-3a52-44bc-bd4c-0ba1b9468272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:25.607077 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffedd6-3a52-44bc-bd4c-0ba1b9468272 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:25.607366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffedd6-3a52-44bc-bd4c-0ba1b9468272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:25.607645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffedd6-3a52-44bc-bd4c-0ba1b9468272 admin admin] PATH_INFO: `/v3/projects/0826357a0e2c46a8bce26a1b0c49bae9/users/366b67832dd74a08855add2f54344f51/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:25.609675 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cffedd6-3a52-44bc-bd4c-0ba1b9468272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0826357a0e2c46a8bce26a1b0c49bae9, user_id=366b67832dd74a08855add2f54344f51, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:25.612392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cffedd6-3a52-44bc-bd4c-0ba1b9468272 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:25.623028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1078/2150] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:25 2022] PUT /identity/v3/projects/0826357a0e2c46a8bce26a1b0c49bae9/users/366b67832dd74a08855add2f54344f51/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:25.634467 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af3160-1bea-4cfc-a076-9fa512edc0e8 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:25.634757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af3160-1bea-4cfc-a076-9fa512edc0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:25.635035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af3160-1bea-4cfc-a076-9fa512edc0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:25.662359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-04af3160-1bea-4cfc-a076-9fa512edc0e8 None None] MFA Rules not processed for user `366b67832dd74a08855add2f54344f51`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:25.664202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-04af3160-1bea-4cfc-a076-9fa512edc0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:25.737831 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1073/2151] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:25.749268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffedd6-3a52-44bc-bd4c-0ba1b9468272 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:25.751504 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3cffedd6-3a52-44bc-bd4c-0ba1b9468272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:25.808595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffedd6-3a52-44bc-bd4c-0ba1b9468272 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:25.812116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262db2f-8ba2-4b44-879c-72ef053411ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:25.813048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262db2f-8ba2-4b44-879c-72ef053411ae service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:25.813333 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262db2f-8ba2-4b44-879c-72ef053411ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:25.813618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262db2f-8ba2-4b44-879c-72ef053411ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:25.823331 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262db2f-8ba2-4b44-879c-72ef053411ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:25.824824 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262db2f-8ba2-4b44-879c-72ef053411ae service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:25.834425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1079/2152] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:25.934989 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af3160-1bea-4cfc-a076-9fa512edc0e8 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:25.945408 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af3160-1bea-4cfc-a076-9fa512edc0e8 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:25.948960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d651fa2-dd65-4ef5-8018-aad3a4b84450 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:25.949606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d651fa2-dd65-4ef5-8018-aad3a4b84450 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:25.949894 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d651fa2-dd65-4ef5-8018-aad3a4b84450 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:25.950170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d651fa2-dd65-4ef5-8018-aad3a4b84450 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:26.014103 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d651fa2-dd65-4ef5-8018-aad3a4b84450 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:26.015814 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d651fa2-dd65-4ef5-8018-aad3a4b84450 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:26.028765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1074/2153] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:28.602648 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262db2f-8ba2-4b44-879c-72ef053411ae service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:28.674667 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262db2f-8ba2-4b44-879c-72ef053411ae service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:28.684596 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb0809-bc3f-4c1a-9612-a7075acdd797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:28.685493 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb0809-bc3f-4c1a-9612-a7075acdd797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:28.685774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb0809-bc3f-4c1a-9612-a7075acdd797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:28.686048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb0809-bc3f-4c1a-9612-a7075acdd797 admin admin] PATH_INFO: `/v3/users/ca03ddc164b94195b40777d0eba0cc11` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:28.687521 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb0809-bc3f-4c1a-9612-a7075acdd797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca03ddc164b94195b40777d0eba0cc11)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:28.689921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb0809-bc3f-4c1a-9612-a7075acdd797 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:28.747678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-38cb0809-bc3f-4c1a-9612-a7075acdd797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca03ddc164b94195b40777d0eba0cc11'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:28.757040 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-38cb0809-bc3f-4c1a-9612-a7075acdd797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca03ddc164b94195b40777d0eba0cc11'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:28.757774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-38cb0809-bc3f-4c1a-9612-a7075acdd797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca03ddc164b94195b40777d0eba0cc11'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:28.765551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-38cb0809-bc3f-4c1a-9612-a7075acdd797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca03ddc164b94195b40777d0eba0cc11'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:28.774033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:28.790494 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-38cb0809-bc3f-4c1a-9612-a7075acdd797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca03ddc164b94195b40777d0eba0cc11'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:28.802245 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1080/2154] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:28 2022] DELETE /identity/v3/users/ca03ddc164b94195b40777d0eba0cc11 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:28.806760 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7865e7-7ef1-4bc0-96e2-6b1e4c487bb2 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:28.807292 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7865e7-7ef1-4bc0-96e2-6b1e4c487bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:28.807292 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7865e7-7ef1-4bc0-96e2-6b1e4c487bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:28.837560 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-cc7865e7-7ef1-4bc0-96e2-6b1e4c487bb2 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:28.839297 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-cc7865e7-7ef1-4bc0-96e2-6b1e4c487bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:28.854404 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:28.976160 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1075/2155] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:28.993135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb0809-bc3f-4c1a-9612-a7075acdd797 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:28.997603 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-38cb0809-bc3f-4c1a-9612-a7075acdd797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:29.105445 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb0809-bc3f-4c1a-9612-a7075acdd797 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:29.117402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46c9dc-432a-4650-b026-2423cda27486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:29.117402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46c9dc-432a-4650-b026-2423cda27486 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:29.117402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46c9dc-432a-4650-b026-2423cda27486 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:29.117402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46c9dc-432a-4650-b026-2423cda27486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:29.124087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46c9dc-432a-4650-b026-2423cda27486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:29.126053 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46c9dc-432a-4650-b026-2423cda27486 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:29.137963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1081/2156] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:29.522209 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7865e7-7ef1-4bc0-96e2-6b1e4c487bb2 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:29.523745 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-cc7865e7-7ef1-4bc0-96e2-6b1e4c487bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:29.538468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7865e7-7ef1-4bc0-96e2-6b1e4c487bb2 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:29.543423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:29.544933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:29.545400 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:29.545772 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6 admin admin] PATH_INFO: `/v3/projects/cd00901619844968a5fe11a39636669b` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:29.547039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd00901619844968a5fe11a39636669b)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:29.549084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:29.599065 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd00901619844968a5fe11a39636669b', 'request_id': 'req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:29.608122 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd00901619844968a5fe11a39636669b', 'request_id': 'req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:29.613835 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd00901619844968a5fe11a39636669b', 'request_id': 'req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:29.624746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6 admin admin] The token cache is being invalidate because project cd00901619844968a5fe11a39636669b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:29.625062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:29.625814 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:29.627358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1076/2157] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:29 2022] DELETE /identity/v3/projects/cd00901619844968a5fe11a39636669b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:29.631369 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46c9dc-432a-4650-b026-2423cda27486 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:29.641398 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46c9dc-432a-4650-b026-2423cda27486 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:29.699373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaebd52-58e3-4409-8e39-27e8878cabef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:29.700248 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaebd52-58e3-4409-8e39-27e8878cabef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:29.700574 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaebd52-58e3-4409-8e39-27e8878cabef admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:29.700854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaebd52-58e3-4409-8e39-27e8878cabef admin admin] PATH_INFO: `/v3/users/366b67832dd74a08855add2f54344f51` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:29.702046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaebd52-58e3-4409-8e39-27e8878cabef admin admin] RBAC: Authorizing `identity:delete_user(user_id=366b67832dd74a08855add2f54344f51)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:29.703973 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaebd52-58e3-4409-8e39-27e8878cabef admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:29.751271 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ccaebd52-58e3-4409-8e39-27e8878cabef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '366b67832dd74a08855add2f54344f51'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:29.759305 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ccaebd52-58e3-4409-8e39-27e8878cabef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '366b67832dd74a08855add2f54344f51'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:29.760161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ccaebd52-58e3-4409-8e39-27e8878cabef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '366b67832dd74a08855add2f54344f51'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:29.767323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ccaebd52-58e3-4409-8e39-27e8878cabef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '366b67832dd74a08855add2f54344f51'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:29.781995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ccaebd52-58e3-4409-8e39-27e8878cabef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '366b67832dd74a08855add2f54344f51'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:29.789851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1082/2158] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:29 2022] DELETE /identity/v3/users/366b67832dd74a08855add2f54344f51 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:30.076738 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:30.078871 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:30.093239 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b79e2-a8b1-4543-9e4f-3f786a65cdd6 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:30.155836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1f9aa-d919-42f4-a393-6c6b0590465e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:30.156656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1f9aa-d919-42f4-a393-6c6b0590465e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:30.156938 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1f9aa-d919-42f4-a393-6c6b0590465e admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:30.157211 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1f9aa-d919-42f4-a393-6c6b0590465e admin admin] PATH_INFO: `/v3/projects/0826357a0e2c46a8bce26a1b0c49bae9` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:30.158519 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1f9aa-d919-42f4-a393-6c6b0590465e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0826357a0e2c46a8bce26a1b0c49bae9)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:30.160509 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1f9aa-d919-42f4-a393-6c6b0590465e admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:30.209160 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:30.213251 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-75f1f9aa-d919-42f4-a393-6c6b0590465e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0826357a0e2c46a8bce26a1b0c49bae9', 'request_id': 'req-75f1f9aa-d919-42f4-a393-6c6b0590465e'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:30.223697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-75f1f9aa-d919-42f4-a393-6c6b0590465e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0826357a0e2c46a8bce26a1b0c49bae9', 'request_id': 'req-75f1f9aa-d919-42f4-a393-6c6b0590465e'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:30.229216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-75f1f9aa-d919-42f4-a393-6c6b0590465e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0826357a0e2c46a8bce26a1b0c49bae9', 'request_id': 'req-75f1f9aa-d919-42f4-a393-6c6b0590465e'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:30.241429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-75f1f9aa-d919-42f4-a393-6c6b0590465e admin admin] The token cache is being invalidate because project 0826357a0e2c46a8bce26a1b0c49bae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:30.241717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-75f1f9aa-d919-42f4-a393-6c6b0590465e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:30.242374 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-75f1f9aa-d919-42f4-a393-6c6b0590465e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:30.244055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1077/2159] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:30 2022] DELETE /identity/v3/projects/0826357a0e2c46a8bce26a1b0c49bae9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:30.248981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf484b8-d406-4797-bbe4-8ca78b376c1b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:30.249339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf484b8-d406-4797-bbe4-8ca78b376c1b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:30.250166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf484b8-d406-4797-bbe4-8ca78b376c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:30.270400 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-daf484b8-d406-4797-bbe4-8ca78b376c1b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:30.271710 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-daf484b8-d406-4797-bbe4-8ca78b376c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:30.276738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:30.329356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1f9aa-d919-42f4-a393-6c6b0590465e admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:30.330775 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-75f1f9aa-d919-42f4-a393-6c6b0590465e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:30.343718 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1083/2160] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:30.354238 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02af28-d18f-4866-a58a-f6107a1fdfe0 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:30.354537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02af28-d18f-4866-a58a-f6107a1fdfe0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:30.354892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02af28-d18f-4866-a58a-f6107a1fdfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:30.358778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1f9aa-d919-42f4-a393-6c6b0590465e admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:30.363563 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d7e1c-09ed-4b75-9550-12234563668b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:30.364468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d7e1c-09ed-4b75-9550-12234563668b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:30.365260 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d7e1c-09ed-4b75-9550-12234563668b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:30.365535 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d7e1c-09ed-4b75-9550-12234563668b admin admin] PATH_INFO: `/v3/users/6e3af9cfd9dd496db158f9a3677b47b4` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:30.366763 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d7e1c-09ed-4b75-9550-12234563668b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e3af9cfd9dd496db158f9a3677b47b4)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:30.368715 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d7e1c-09ed-4b75-9550-12234563668b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:30.374969 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ed02af28-d18f-4866-a58a-f6107a1fdfe0 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:30.376170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ed02af28-d18f-4866-a58a-f6107a1fdfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:30.388616 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1084/2161] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:30.392734 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02af28-d18f-4866-a58a-f6107a1fdfe0 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:30.403016 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02af28-d18f-4866-a58a-f6107a1fdfe0 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:30.408399 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4935ba4-e910-4d3a-9f8d-03979dfb34da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:30.409085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4935ba4-e910-4d3a-9f8d-03979dfb34da admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:30.409366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4935ba4-e910-4d3a-9f8d-03979dfb34da admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:30.409626 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4935ba4-e910-4d3a-9f8d-03979dfb34da admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:30.410497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4935ba4-e910-4d3a-9f8d-03979dfb34da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:30.410834 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4935ba4-e910-4d3a-9f8d-03979dfb34da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:30.412824 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4935ba4-e910-4d3a-9f8d-03979dfb34da admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:30.425586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1085/2162] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:25:30.430539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4935ba4-e910-4d3a-9f8d-03979dfb34da admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:30.435898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-dd3d7e1c-09ed-4b75-9550-12234563668b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e3af9cfd9dd496db158f9a3677b47b4'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:30.444453 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4935ba4-e910-4d3a-9f8d-03979dfb34da admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:30.446697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-dd3d7e1c-09ed-4b75-9550-12234563668b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e3af9cfd9dd496db158f9a3677b47b4'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:30.449846 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec82071-7bdc-485c-be1d-1b7ee869996e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:30.450540 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec82071-7bdc-485c-be1d-1b7ee869996e admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:30.450826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec82071-7bdc-485c-be1d-1b7ee869996e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:30.451103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec82071-7bdc-485c-be1d-1b7ee869996e admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:30.451795 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec82071-7bdc-485c-be1d-1b7ee869996e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:30.453858 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec82071-7bdc-485c-be1d-1b7ee869996e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:30.461239 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-dd3d7e1c-09ed-4b75-9550-12234563668b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e3af9cfd9dd496db158f9a3677b47b4'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:30.474157 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-dd3d7e1c-09ed-4b75-9550-12234563668b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e3af9cfd9dd496db158f9a3677b47b4'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:30.475108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-dd3d7e1c-09ed-4b75-9550-12234563668b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e3af9cfd9dd496db158f9a3677b47b4'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:30.482175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1086/2163] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:30 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:30.487505 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1078/2164] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:30 2022] DELETE /identity/v3/users/6e3af9cfd9dd496db158f9a3677b47b4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:30.487667 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec82071-7bdc-485c-be1d-1b7ee869996e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:30.488387 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cec82071-7bdc-485c-be1d-1b7ee869996e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:30.489410 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de324ff9-6958-4a58-aeb7-bab3fbb12927 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:30.490325 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de324ff9-6958-4a58-aeb7-bab3fbb12927 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:30.490706 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de324ff9-6958-4a58-aeb7-bab3fbb12927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:30.510618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-de324ff9-6958-4a58-aeb7-bab3fbb12927 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:30.512834 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-de324ff9-6958-4a58-aeb7-bab3fbb12927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:30.561032 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec82071-7bdc-485c-be1d-1b7ee869996e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:30.577242 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1079/2165] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:30.587477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de324ff9-6958-4a58-aeb7-bab3fbb12927 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:30.588896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-de324ff9-6958-4a58-aeb7-bab3fbb12927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:30.622133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b73ddf-6c21-4afb-89d3-dd1010d9a35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:30.622808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b73ddf-6c21-4afb-89d3-dd1010d9a35d admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:30.623096 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b73ddf-6c21-4afb-89d3-dd1010d9a35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:30.623372 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b73ddf-6c21-4afb-89d3-dd1010d9a35d admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:30.624144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b73ddf-6c21-4afb-89d3-dd1010d9a35d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:30.626226 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b73ddf-6c21-4afb-89d3-dd1010d9a35d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:30.637099 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:30.648148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1087/2166] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:30 2022] POST /identity/v3/users => generated 291 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:25:30.652185 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b73ddf-6c21-4afb-89d3-dd1010d9a35d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:30.652938 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de324ff9-6958-4a58-aeb7-bab3fbb12927 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:30.657595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1868e40-14d0-4a72-8cb4-f086399f89c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:30.658250 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1868e40-14d0-4a72-8cb4-f086399f89c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:30.658587 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1868e40-14d0-4a72-8cb4-f086399f89c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:30.658804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1868e40-14d0-4a72-8cb4-f086399f89c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:30.659054 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:30.663712 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b73ddf-6c21-4afb-89d3-dd1010d9a35d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:30.669740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fb461-3050-4322-bc89-e9d273a7b3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:30.669927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1868e40-14d0-4a72-8cb4-f086399f89c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:30.670397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fb461-3050-4322-bc89-e9d273a7b3ac admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:30.670676 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fb461-3050-4322-bc89-e9d273a7b3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:30.670950 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fb461-3050-4322-bc89-e9d273a7b3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:30.671329 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1868e40-14d0-4a72-8cb4-f086399f89c5 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:30.671673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fb461-3050-4322-bc89-e9d273a7b3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:30.674756 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fb461-3050-4322-bc89-e9d273a7b3ac admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:30.683193 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1080/2167] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:25:30.686549 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1088/2168] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:25:30.690860 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1868e40-14d0-4a72-8cb4-f086399f89c5 service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:30.701375 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1868e40-14d0-4a72-8cb4-f086399f89c5 service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:30.706123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b921c77-aa9d-4c9c-9843-39b99847bb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:30.706792 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b921c77-aa9d-4c9c-9843-39b99847bb7e admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:30.707081 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b921c77-aa9d-4c9c-9843-39b99847bb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:30.707365 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b921c77-aa9d-4c9c-9843-39b99847bb7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:30.708098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b921c77-aa9d-4c9c-9843-39b99847bb7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:30.709751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b921c77-aa9d-4c9c-9843-39b99847bb7e admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:30.721012 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1081/2169] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:25:30.725639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fb461-3050-4322-bc89-e9d273a7b3ac admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:30.735874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fb461-3050-4322-bc89-e9d273a7b3ac admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:30.742625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540f7a9-9e7b-4f13-9742-bc7f182d9f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:30.743557 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540f7a9-9e7b-4f13-9742-bc7f182d9f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:30.743844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540f7a9-9e7b-4f13-9742-bc7f182d9f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:30.744121 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540f7a9-9e7b-4f13-9742-bc7f182d9f54 admin admin] PATH_INFO: `/v3/projects/99b197a51c8b42339af0a22b5dc563d2/users/aca798b5db5d4dd5a55b2bcefd7ec370/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:30.764692 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540f7a9-9e7b-4f13-9742-bc7f182d9f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b197a51c8b42339af0a22b5dc563d2, user_id=aca798b5db5d4dd5a55b2bcefd7ec370, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:30.768382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540f7a9-9e7b-4f13-9742-bc7f182d9f54 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:30.782943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1089/2170] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:30 2022] PUT /identity/v3/projects/99b197a51c8b42339af0a22b5dc563d2/users/aca798b5db5d4dd5a55b2bcefd7ec370/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:30.793956 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36ea45-ad2c-4ff2-8a15-9aef723a65ae None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:30.794247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36ea45-ad2c-4ff2-8a15-9aef723a65ae None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:30.794538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36ea45-ad2c-4ff2-8a15-9aef723a65ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:30.829126 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4f36ea45-ad2c-4ff2-8a15-9aef723a65ae None None] MFA Rules not processed for user `aca798b5db5d4dd5a55b2bcefd7ec370`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:30.830370 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4f36ea45-ad2c-4ff2-8a15-9aef723a65ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:30.917421 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1082/2171] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:30.920425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540f7a9-9e7b-4f13-9742-bc7f182d9f54 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:30.975453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36ea45-ad2c-4ff2-8a15-9aef723a65ae None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:30.976484 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4f36ea45-ad2c-4ff2-8a15-9aef723a65ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:31.001734 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540f7a9-9e7b-4f13-9742-bc7f182d9f54 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:31.006868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8750fe1-516b-48ee-bc92-d8a04490fc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:31.007635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8750fe1-516b-48ee-bc92-d8a04490fc7f admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.007957 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8750fe1-516b-48ee-bc92-d8a04490fc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.008366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8750fe1-516b-48ee-bc92-d8a04490fc7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:31.009102 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8750fe1-516b-48ee-bc92-d8a04490fc7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:31.011178 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8750fe1-516b-48ee-bc92-d8a04490fc7f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:31.044165 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1090/2172] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:30 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:31.045503 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8750fe1-516b-48ee-bc92-d8a04490fc7f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:31.049428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36ea45-ad2c-4ff2-8a15-9aef723a65ae None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:31.108455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2258ba-b0ad-47b4-982a-7ab2b062ee9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:31.110067 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2258ba-b0ad-47b4-982a-7ab2b062ee9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.110345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2258ba-b0ad-47b4-982a-7ab2b062ee9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.110617 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2258ba-b0ad-47b4-982a-7ab2b062ee9b admin admin] PATH_INFO: `/v3/projects/1652b95123c449b7b00990484330972c` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:31.111684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8750fe1-516b-48ee-bc92-d8a04490fc7f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:31.111867 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2258ba-b0ad-47b4-982a-7ab2b062ee9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1652b95123c449b7b00990484330972c)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:31.114041 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2258ba-b0ad-47b4-982a-7ab2b062ee9b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:31.117218 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65384a58-9b1f-4591-a411-66257e1e482d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:31.117891 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65384a58-9b1f-4591-a411-66257e1e482d admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.118315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65384a58-9b1f-4591-a411-66257e1e482d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.118598 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65384a58-9b1f-4591-a411-66257e1e482d admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:31.119294 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65384a58-9b1f-4591-a411-66257e1e482d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:31.121741 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65384a58-9b1f-4591-a411-66257e1e482d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:31.130410 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:31.145692 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1091/2173] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:31 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:31.150417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65384a58-9b1f-4591-a411-66257e1e482d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:31.161020 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65384a58-9b1f-4591-a411-66257e1e482d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:31.178757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-db2258ba-b0ad-47b4-982a-7ab2b062ee9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1652b95123c449b7b00990484330972c', 'request_id': 'req-db2258ba-b0ad-47b4-982a-7ab2b062ee9b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:31.190535 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-db2258ba-b0ad-47b4-982a-7ab2b062ee9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1652b95123c449b7b00990484330972c', 'request_id': 'req-db2258ba-b0ad-47b4-982a-7ab2b062ee9b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:31.197588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-db2258ba-b0ad-47b4-982a-7ab2b062ee9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1652b95123c449b7b00990484330972c', 'request_id': 'req-db2258ba-b0ad-47b4-982a-7ab2b062ee9b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:31.209540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-db2258ba-b0ad-47b4-982a-7ab2b062ee9b admin admin] The token cache is being invalidate because project 1652b95123c449b7b00990484330972c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:31.209833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-db2258ba-b0ad-47b4-982a-7ab2b062ee9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:31.210508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-db2258ba-b0ad-47b4-982a-7ab2b062ee9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:31.211201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:31.212498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1083/2174] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:30 2022] DELETE /identity/v3/projects/1652b95123c449b7b00990484330972c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:31.217905 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c671e-7b48-4e65-9ad8-cbd7f26c1674 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.218200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c671e-7b48-4e65-9ad8-cbd7f26c1674 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.218476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c671e-7b48-4e65-9ad8-cbd7f26c1674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:31.233813 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d141de-a9e5-44c6-bd0e-53736ac34685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:31.235194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d141de-a9e5-44c6-bd0e-53736ac34685 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.235670 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d141de-a9e5-44c6-bd0e-53736ac34685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.236102 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d141de-a9e5-44c6-bd0e-53736ac34685 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:31.237023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d141de-a9e5-44c6-bd0e-53736ac34685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:31.238798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d141de-a9e5-44c6-bd0e-53736ac34685 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:31.240895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-ef6c671e-7b48-4e65-9ad8-cbd7f26c1674 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:31.242249 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ef6c671e-7b48-4e65-9ad8-cbd7f26c1674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:31.249910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1092/2175] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:25:31.257979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1084/2176] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:31.258176 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d141de-a9e5-44c6-bd0e-53736ac34685 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:31.261732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-04d141de-a9e5-44c6-bd0e-53736ac34685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:31.261999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007712f4-dc71-42ef-8808-33ffb625a33f None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.262244 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007712f4-dc71-42ef-8808-33ffb625a33f None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.262526 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007712f4-dc71-42ef-8808-33ffb625a33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:31.281038 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d141de-a9e5-44c6-bd0e-53736ac34685 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:31.283881 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-007712f4-dc71-42ef-8808-33ffb625a33f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:31.285561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-007712f4-dc71-42ef-8808-33ffb625a33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:31.286378 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b72a5-bfaa-407f-9da5-fe2f94dd6ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:31.287851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b72a5-bfaa-407f-9da5-fe2f94dd6ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.288195 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b72a5-bfaa-407f-9da5-fe2f94dd6ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.288439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b72a5-bfaa-407f-9da5-fe2f94dd6ce7 admin admin] PATH_INFO: `/v3/projects/d40d5b5e50c146bb86710c33cd45e49d/users/c33e28ae88674c828fbbfb556010d34f/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:31.301167 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1085/2177] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:31.301789 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007712f4-dc71-42ef-8808-33ffb625a33f None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:31.311696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007712f4-dc71-42ef-8808-33ffb625a33f None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:31.317723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96409d17-5e33-4ea5-9460-1eff0e685ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:31.318757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96409d17-5e33-4ea5-9460-1eff0e685ecb admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.319039 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96409d17-5e33-4ea5-9460-1eff0e685ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.319313 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96409d17-5e33-4ea5-9460-1eff0e685ecb admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:31.320055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96409d17-5e33-4ea5-9460-1eff0e685ecb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:31.320412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96409d17-5e33-4ea5-9460-1eff0e685ecb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:31.322225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96409d17-5e33-4ea5-9460-1eff0e685ecb admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:31.326278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b72a5-bfaa-407f-9da5-fe2f94dd6ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d40d5b5e50c146bb86710c33cd45e49d, user_id=c33e28ae88674c828fbbfb556010d34f, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:31.330087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b72a5-bfaa-407f-9da5-fe2f94dd6ce7 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:31.344986 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1086/2178] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:25:31.351021 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96409d17-5e33-4ea5-9460-1eff0e685ecb admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:31.363265 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1093/2179] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:31 2022] PUT /identity/v3/projects/d40d5b5e50c146bb86710c33cd45e49d/users/c33e28ae88674c828fbbfb556010d34f/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:31.367384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b72a5-bfaa-407f-9da5-fe2f94dd6ce7 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:31.371288 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96409d17-5e33-4ea5-9460-1eff0e685ecb admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:31.461227 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b72a5-bfaa-407f-9da5-fe2f94dd6ce7 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:31.466673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734ac83-440d-44ef-8e0a-e620178ef9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:31.467336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734ac83-440d-44ef-8e0a-e620178ef9ff admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.468172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d0752-773e-4c45-991f-c259cd9a8c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:31.468541 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734ac83-440d-44ef-8e0a-e620178ef9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.469070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d0752-773e-4c45-991f-c259cd9a8c6f admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.469412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d0752-773e-4c45-991f-c259cd9a8c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.469748 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d0752-773e-4c45-991f-c259cd9a8c6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:31.470647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d0752-773e-4c45-991f-c259cd9a8c6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:31.472866 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734ac83-440d-44ef-8e0a-e620178ef9ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:31.472979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d0752-773e-4c45-991f-c259cd9a8c6f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:31.473858 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6734ac83-440d-44ef-8e0a-e620178ef9ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:31.476096 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6734ac83-440d-44ef-8e0a-e620178ef9ff admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:31.501533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1094/2180] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:25:31.506005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734ac83-440d-44ef-8e0a-e620178ef9ff admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:31.506459 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1087/2181] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:31 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:31.510209 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d0752-773e-4c45-991f-c259cd9a8c6f admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:31.517849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734ac83-440d-44ef-8e0a-e620178ef9ff admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:31.592630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36370e-8204-4175-871b-eee9a85c3f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:31.593528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36370e-8204-4175-871b-eee9a85c3f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.593839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36370e-8204-4175-871b-eee9a85c3f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.594113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36370e-8204-4175-871b-eee9a85c3f81 admin admin] PATH_INFO: `/v3/projects/d40d5b5e50c146bb86710c33cd45e49d/users/c33e28ae88674c828fbbfb556010d34f/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:31.596981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36370e-8204-4175-871b-eee9a85c3f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d40d5b5e50c146bb86710c33cd45e49d, user_id=c33e28ae88674c828fbbfb556010d34f, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:31.600407 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36370e-8204-4175-871b-eee9a85c3f81 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:31.602237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d0752-773e-4c45-991f-c259cd9a8c6f admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:31.609791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1ad73-0f87-4713-af10-15580e2dfd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:31.610565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1ad73-0f87-4713-af10-15580e2dfd96 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.610959 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1ad73-0f87-4713-af10-15580e2dfd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.611302 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1ad73-0f87-4713-af10-15580e2dfd96 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:31.620938 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a1ad73-0f87-4713-af10-15580e2dfd96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:31.621194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1095/2182] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:31 2022] PUT /identity/v3/projects/d40d5b5e50c146bb86710c33cd45e49d/users/c33e28ae88674c828fbbfb556010d34f/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:31.621298 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a1ad73-0f87-4713-af10-15580e2dfd96 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:31.625020 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f8750-b73c-4d9d-9113-81729987b5bf None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.629244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f8750-b73c-4d9d-9113-81729987b5bf None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.629244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f8750-b73c-4d9d-9113-81729987b5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:31.633690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:31.652700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1088/2183] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:31 2022] POST /identity/v3/users => generated 291 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:31.656905 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1ad73-0f87-4713-af10-15580e2dfd96 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:31.669372 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1ad73-0f87-4713-af10-15580e2dfd96 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:31.674597 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e27f8750-b73c-4d9d-9113-81729987b5bf None None] MFA Rules not processed for user `c33e28ae88674c828fbbfb556010d34f`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:31.675780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e27f8750-b73c-4d9d-9113-81729987b5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:31.732126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:31.766361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7288cd-5520-4713-b30a-55ef993fb981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:31.767316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7288cd-5520-4713-b30a-55ef993fb981 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.770631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7288cd-5520-4713-b30a-55ef993fb981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.772811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7288cd-5520-4713-b30a-55ef993fb981 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:31.773842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7288cd-5520-4713-b30a-55ef993fb981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:31.776125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7288cd-5520-4713-b30a-55ef993fb981 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:31.795825 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1089/2184] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:31.804630 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7288cd-5520-4713-b30a-55ef993fb981 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:31.816173 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7288cd-5520-4713-b30a-55ef993fb981 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:31.816435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1096/2185] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:31.838547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2bb1b-497b-419e-a6d0-b704a7a7e7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:31.840212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2bb1b-497b-419e-a6d0-b704a7a7e7bd admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.840720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2bb1b-497b-419e-a6d0-b704a7a7e7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.841152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2bb1b-497b-419e-a6d0-b704a7a7e7bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:31.842064 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2bb1b-497b-419e-a6d0-b704a7a7e7bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:31.843883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2bb1b-497b-419e-a6d0-b704a7a7e7bd admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:31.846737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f8750-b73c-4d9d-9113-81729987b5bf None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:31.851252 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e27f8750-b73c-4d9d-9113-81729987b5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:31.859917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1090/2186] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:25:31.863908 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2bb1b-497b-419e-a6d0-b704a7a7e7bd admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:31.885510 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2bb1b-497b-419e-a6d0-b704a7a7e7bd admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:31.891868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881212e2-4cbd-4cf1-9464-5a9755535afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:31.894144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881212e2-4cbd-4cf1-9464-5a9755535afc admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.894970 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881212e2-4cbd-4cf1-9464-5a9755535afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.895417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881212e2-4cbd-4cf1-9464-5a9755535afc admin admin] PATH_INFO: `/v3/projects/fac1b4711d514d53b1c08c841ee0af2f/users/9a1c8ca8857844919445d265b64d2192/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:31.934991 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881212e2-4cbd-4cf1-9464-5a9755535afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fac1b4711d514d53b1c08c841ee0af2f, user_id=9a1c8ca8857844919445d265b64d2192, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:31.939640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881212e2-4cbd-4cf1-9464-5a9755535afc admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:31.959329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f8750-b73c-4d9d-9113-81729987b5bf None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:31.961069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1091/2187] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:31 2022] PUT /identity/v3/projects/fac1b4711d514d53b1c08c841ee0af2f/users/9a1c8ca8857844919445d265b64d2192/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:31.976251 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d644f-1c11-4b6a-92c2-42b15742e687 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:31.976584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d644f-1c11-4b6a-92c2-42b15742e687 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:31.976927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d644f-1c11-4b6a-92c2-42b15742e687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:32.007603 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-951d644f-1c11-4b6a-92c2-42b15742e687 None None] MFA Rules not processed for user `9a1c8ca8857844919445d265b64d2192`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:32.008866 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-951d644f-1c11-4b6a-92c2-42b15742e687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:32.018703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ddefc-ebae-4cda-87fe-85ae3df77edb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:32.019664 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ddefc-ebae-4cda-87fe-85ae3df77edb service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:32.019964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ddefc-ebae-4cda-87fe-85ae3df77edb service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:32.020240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ddefc-ebae-4cda-87fe-85ae3df77edb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:32.090045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1092/2188] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:32.102376 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d644f-1c11-4b6a-92c2-42b15742e687 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:32.104837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-951d644f-1c11-4b6a-92c2-42b15742e687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:32.108281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ddefc-ebae-4cda-87fe-85ae3df77edb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:32.109974 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ddefc-ebae-4cda-87fe-85ae3df77edb service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:32.117832 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d644f-1c11-4b6a-92c2-42b15742e687 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:32.121057 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1097/2189] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:32.124419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74df3b-f7ef-4cd1-9eb1-de484bb6eaa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:32.125083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74df3b-f7ef-4cd1-9eb1-de484bb6eaa5 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:32.125368 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74df3b-f7ef-4cd1-9eb1-de484bb6eaa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:32.125647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74df3b-f7ef-4cd1-9eb1-de484bb6eaa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:32.137259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74df3b-f7ef-4cd1-9eb1-de484bb6eaa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:32.138772 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74df3b-f7ef-4cd1-9eb1-de484bb6eaa5 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:32.152763 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1093/2190] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:25:32.543872 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ddefc-ebae-4cda-87fe-85ae3df77edb service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:32.603636 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:32.627097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ddefc-ebae-4cda-87fe-85ae3df77edb service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:32.632574 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4246bf-6d92-4bf6-83ef-2b16019370f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:32.633397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4246bf-6d92-4bf6-83ef-2b16019370f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:32.633705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4246bf-6d92-4bf6-83ef-2b16019370f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:32.633980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4246bf-6d92-4bf6-83ef-2b16019370f9 admin admin] PATH_INFO: `/v3/users/aca798b5db5d4dd5a55b2bcefd7ec370` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:32.635192 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4246bf-6d92-4bf6-83ef-2b16019370f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aca798b5db5d4dd5a55b2bcefd7ec370)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:32.637157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4246bf-6d92-4bf6-83ef-2b16019370f9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:32.684438 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4e4246bf-6d92-4bf6-83ef-2b16019370f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aca798b5db5d4dd5a55b2bcefd7ec370'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:32.693989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4e4246bf-6d92-4bf6-83ef-2b16019370f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aca798b5db5d4dd5a55b2bcefd7ec370'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:32.694605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4e4246bf-6d92-4bf6-83ef-2b16019370f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aca798b5db5d4dd5a55b2bcefd7ec370'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:32.702202 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4e4246bf-6d92-4bf6-83ef-2b16019370f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aca798b5db5d4dd5a55b2bcefd7ec370'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:32.714383 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4e4246bf-6d92-4bf6-83ef-2b16019370f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aca798b5db5d4dd5a55b2bcefd7ec370'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:32.721117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1098/2191] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:32 2022] DELETE /identity/v3/users/aca798b5db5d4dd5a55b2bcefd7ec370 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:32.727238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7183fb-ac20-4919-a185-25f2cc579717 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:32.727625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7183fb-ac20-4919-a185-25f2cc579717 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:32.727968 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7183fb-ac20-4919-a185-25f2cc579717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:32.746669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-8b7183fb-ac20-4919-a185-25f2cc579717 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:32.748322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-8b7183fb-ac20-4919-a185-25f2cc579717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:32.775852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:32.823399 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1094/2192] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:32.832950 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4246bf-6d92-4bf6-83ef-2b16019370f9 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:32.834464 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4e4246bf-6d92-4bf6-83ef-2b16019370f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:32.856615 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76fd1-a55e-4b43-9b25-d8d4d3416cdb None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:32.856912 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76fd1-a55e-4b43-9b25-d8d4d3416cdb None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:32.857196 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76fd1-a55e-4b43-9b25-d8d4d3416cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:32.880766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-dab76fd1-a55e-4b43-9b25-d8d4d3416cdb None None] MFA Rules not processed for user `83ba05eb66ec44598ec0809cfc0c4ccd`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:32.882403 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-dab76fd1-a55e-4b43-9b25-d8d4d3416cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:32.935381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4246bf-6d92-4bf6-83ef-2b16019370f9 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:32.940668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba6554-993b-4601-aaf1-76541cc5a0c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:32.941341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba6554-993b-4601-aaf1-76541cc5a0c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:32.941621 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba6554-993b-4601-aaf1-76541cc5a0c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:32.941897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba6554-993b-4601-aaf1-76541cc5a0c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:32.955066 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba6554-993b-4601-aaf1-76541cc5a0c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:32.956772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba6554-993b-4601-aaf1-76541cc5a0c1 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:32.966149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1095/2193] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:32.967506 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1099/2194] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:33.403165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab76fd1-a55e-4b43-9b25-d8d4d3416cdb None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:33.404746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-dab76fd1-a55e-4b43-9b25-d8d4d3416cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:33.430127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab76fd1-a55e-4b43-9b25-d8d4d3416cdb None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:33.436111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:33.437123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:33.437732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:33.438068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c admin admin] PATH_INFO: `/v3/projects/99b197a51c8b42339af0a22b5dc563d2` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:33.439331 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b197a51c8b42339af0a22b5dc563d2)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:33.441353 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:33.494508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b197a51c8b42339af0a22b5dc563d2', 'request_id': 'req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:33.502753 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b197a51c8b42339af0a22b5dc563d2', 'request_id': 'req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:33.516246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b197a51c8b42339af0a22b5dc563d2', 'request_id': 'req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:33.539685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c admin admin] The token cache is being invalidate because project 99b197a51c8b42339af0a22b5dc563d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:33.540453 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:33.541028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:33.543657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1096/2195] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:33 2022] DELETE /identity/v3/projects/99b197a51c8b42339af0a22b5dc563d2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:33.547604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba6554-993b-4601-aaf1-76541cc5a0c1 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:33.563210 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba6554-993b-4601-aaf1-76541cc5a0c1 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:33.647187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd633f-9054-440e-b1af-cb47fbe4712c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:33.648065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd633f-9054-440e-b1af-cb47fbe4712c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:33.648432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd633f-9054-440e-b1af-cb47fbe4712c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:33.648709 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd633f-9054-440e-b1af-cb47fbe4712c admin admin] PATH_INFO: `/v3/users/c33e28ae88674c828fbbfb556010d34f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:33.656165 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd633f-9054-440e-b1af-cb47fbe4712c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c33e28ae88674c828fbbfb556010d34f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:33.658624 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd633f-9054-440e-b1af-cb47fbe4712c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:33.662427 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:33.746950 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5edd633f-9054-440e-b1af-cb47fbe4712c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c33e28ae88674c828fbbfb556010d34f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:33.759043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5edd633f-9054-440e-b1af-cb47fbe4712c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c33e28ae88674c828fbbfb556010d34f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:33.759698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5edd633f-9054-440e-b1af-cb47fbe4712c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c33e28ae88674c828fbbfb556010d34f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:33.770814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5edd633f-9054-440e-b1af-cb47fbe4712c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c33e28ae88674c828fbbfb556010d34f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:33.787797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5edd633f-9054-440e-b1af-cb47fbe4712c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c33e28ae88674c828fbbfb556010d34f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:33.794118 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1100/2196] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:33 2022] DELETE /identity/v3/users/c33e28ae88674c828fbbfb556010d34f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:34.098340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:34.100876 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:34.115333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160a78c-c6a4-426d-aeac-0b4e8cd67e5c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:34.170239 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:34.177516 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47707d8b-b847-48a8-be2d-f25fd2bd4293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:34.178187 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47707d8b-b847-48a8-be2d-f25fd2bd4293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:34.178727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47707d8b-b847-48a8-be2d-f25fd2bd4293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:34.179697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47707d8b-b847-48a8-be2d-f25fd2bd4293 admin admin] PATH_INFO: `/v3/projects/d40d5b5e50c146bb86710c33cd45e49d` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:34.181007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47707d8b-b847-48a8-be2d-f25fd2bd4293 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d40d5b5e50c146bb86710c33cd45e49d)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:34.183138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47707d8b-b847-48a8-be2d-f25fd2bd4293 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:34.232237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-47707d8b-b847-48a8-be2d-f25fd2bd4293 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd40d5b5e50c146bb86710c33cd45e49d', 'request_id': 'req-47707d8b-b847-48a8-be2d-f25fd2bd4293'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:34.240070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-47707d8b-b847-48a8-be2d-f25fd2bd4293 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd40d5b5e50c146bb86710c33cd45e49d', 'request_id': 'req-47707d8b-b847-48a8-be2d-f25fd2bd4293'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:34.245641 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-47707d8b-b847-48a8-be2d-f25fd2bd4293 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd40d5b5e50c146bb86710c33cd45e49d', 'request_id': 'req-47707d8b-b847-48a8-be2d-f25fd2bd4293'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:34.257128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-47707d8b-b847-48a8-be2d-f25fd2bd4293 admin admin] The token cache is being invalidate because project d40d5b5e50c146bb86710c33cd45e49d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:34.257415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-47707d8b-b847-48a8-be2d-f25fd2bd4293 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:34.258073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-47707d8b-b847-48a8-be2d-f25fd2bd4293 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:34.259688 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1097/2197] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:34 2022] DELETE /identity/v3/projects/d40d5b5e50c146bb86710c33cd45e49d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:34.264995 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4842ee-1bc3-460b-a76f-ce0bc5864896 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:34.265286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4842ee-1bc3-460b-a76f-ce0bc5864896 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:34.265561 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4842ee-1bc3-460b-a76f-ce0bc5864896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:34.284771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5e4842ee-1bc3-460b-a76f-ce0bc5864896 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:34.285926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5e4842ee-1bc3-460b-a76f-ce0bc5864896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:34.351970 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1101/2198] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:34.359585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903f5a-da34-4972-ad18-bbfba00dc0a2 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:34.359875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903f5a-da34-4972-ad18-bbfba00dc0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:34.360155 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903f5a-da34-4972-ad18-bbfba00dc0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:34.378846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-1c903f5a-da34-4972-ad18-bbfba00dc0a2 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:34.380065 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-1c903f5a-da34-4972-ad18-bbfba00dc0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:34.389093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1098/2199] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:34.393658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4842ee-1bc3-460b-a76f-ce0bc5864896 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:34.405199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4842ee-1bc3-460b-a76f-ce0bc5864896 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:34.410334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d86bd-9bd1-4265-9cd1-29e44a33384f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:34.411028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d86bd-9bd1-4265-9cd1-29e44a33384f admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:34.411339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d86bd-9bd1-4265-9cd1-29e44a33384f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:34.411654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d86bd-9bd1-4265-9cd1-29e44a33384f admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:34.412495 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d86bd-9bd1-4265-9cd1-29e44a33384f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:34.412807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d86bd-9bd1-4265-9cd1-29e44a33384f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:34.414516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d86bd-9bd1-4265-9cd1-29e44a33384f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:34.431628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1102/2200] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:25:34.433050 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c903f5a-da34-4972-ad18-bbfba00dc0a2 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:34.443872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c903f5a-da34-4972-ad18-bbfba00dc0a2 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:34.448836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc86695-61fc-44d0-a2bf-6351f9a4480b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:34.449517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc86695-61fc-44d0-a2bf-6351f9a4480b admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:34.449803 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc86695-61fc-44d0-a2bf-6351f9a4480b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:34.450079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc86695-61fc-44d0-a2bf-6351f9a4480b admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:34.450792 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc86695-61fc-44d0-a2bf-6351f9a4480b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:34.452746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc86695-61fc-44d0-a2bf-6351f9a4480b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:34.481892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1099/2201] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:34 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:34.486121 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d86bd-9bd1-4265-9cd1-29e44a33384f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:34.497085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d86bd-9bd1-4265-9cd1-29e44a33384f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:34.557345 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c7fbc-f422-43b4-895f-4be9b6f830d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:34.558026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c7fbc-f422-43b4-895f-4be9b6f830d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:34.558311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c7fbc-f422-43b4-895f-4be9b6f830d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:34.558586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c7fbc-f422-43b4-895f-4be9b6f830d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:34.559290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c7fbc-f422-43b4-895f-4be9b6f830d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:34.561331 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c7fbc-f422-43b4-895f-4be9b6f830d0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:34.571980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:34.586457 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1103/2202] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:34 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:34.590183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc86695-61fc-44d0-a2bf-6351f9a4480b admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:34.601034 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc86695-61fc-44d0-a2bf-6351f9a4480b admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:34.605662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680633c-9d21-4cd7-a0d2-ec5d4efbcaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:34.606408 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680633c-9d21-4cd7-a0d2-ec5d4efbcaad admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:34.606755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680633c-9d21-4cd7-a0d2-ec5d4efbcaad admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:34.607094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680633c-9d21-4cd7-a0d2-ec5d4efbcaad admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:34.607886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680633c-9d21-4cd7-a0d2-ec5d4efbcaad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:34.609568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680633c-9d21-4cd7-a0d2-ec5d4efbcaad admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:34.624545 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1100/2203] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:34.628697 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c7fbc-f422-43b4-895f-4be9b6f830d0 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:34.639296 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c7fbc-f422-43b4-895f-4be9b6f830d0 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:34.643952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d31a4-eb61-440d-91a7-9a0f92e8720f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:34.644715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d31a4-eb61-440d-91a7-9a0f92e8720f admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:34.645002 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d31a4-eb61-440d-91a7-9a0f92e8720f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:34.645275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d31a4-eb61-440d-91a7-9a0f92e8720f admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:34.646093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d31a4-eb61-440d-91a7-9a0f92e8720f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:34.648173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d31a4-eb61-440d-91a7-9a0f92e8720f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:34.659288 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1104/2204] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:34.662797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680633c-9d21-4cd7-a0d2-ec5d4efbcaad admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:34.672874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680633c-9d21-4cd7-a0d2-ec5d4efbcaad admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:34.677865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc3001f-838d-446e-bfd6-9d3bcebe8509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:34.678723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3001f-838d-446e-bfd6-9d3bcebe8509 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:34.679122 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3001f-838d-446e-bfd6-9d3bcebe8509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:34.679401 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3001f-838d-446e-bfd6-9d3bcebe8509 admin admin] PATH_INFO: `/v3/projects/4f13b0a634b8423581a33819b24a1ffc/users/108789b84d30459ebc9b3e916f2f96a0/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:34.695200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc3001f-838d-446e-bfd6-9d3bcebe8509 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f13b0a634b8423581a33819b24a1ffc, user_id=108789b84d30459ebc9b3e916f2f96a0, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:34.697926 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc3001f-838d-446e-bfd6-9d3bcebe8509 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:34.715488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1101/2205] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:34 2022] PUT /identity/v3/projects/4f13b0a634b8423581a33819b24a1ffc/users/108789b84d30459ebc9b3e916f2f96a0/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:34.720888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0dc04-8b5c-4028-9620-db27242eb9f0 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:34.721173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0dc04-8b5c-4028-9620-db27242eb9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:34.721449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0dc04-8b5c-4028-9620-db27242eb9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:34.740432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8fc0dc04-8b5c-4028-9620-db27242eb9f0 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:34.741655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8fc0dc04-8b5c-4028-9620-db27242eb9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:34.806258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1105/2206] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:34.815416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc3001f-838d-446e-bfd6-9d3bcebe8509 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:34.817808 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-dbc3001f-838d-446e-bfd6-9d3bcebe8509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:34.883982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc3001f-838d-446e-bfd6-9d3bcebe8509 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:34.887686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c607acb-57ed-4446-aa51-bf5fa0a3836c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:34.888375 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c607acb-57ed-4446-aa51-bf5fa0a3836c service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:34.888664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c607acb-57ed-4446-aa51-bf5fa0a3836c service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:34.888944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c607acb-57ed-4446-aa51-bf5fa0a3836c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:34.898687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c607acb-57ed-4446-aa51-bf5fa0a3836c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:34.900119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c607acb-57ed-4446-aa51-bf5fa0a3836c service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:34.909921 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1102/2207] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:35.529199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984862c-02d9-4347-b5fa-41586eb982a0 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:35.529516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984862c-02d9-4347-b5fa-41586eb982a0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:35.529793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984862c-02d9-4347-b5fa-41586eb982a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:35.556681 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f984862c-02d9-4347-b5fa-41586eb982a0 None None] MFA Rules not processed for user `108789b84d30459ebc9b3e916f2f96a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:35.557998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f984862c-02d9-4347-b5fa-41586eb982a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:35.626260 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1106/2208] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:35.630520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c607acb-57ed-4446-aa51-bf5fa0a3836c service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:35.641113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c607acb-57ed-4446-aa51-bf5fa0a3836c service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:35.645921 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb3e28-15e7-47a1-a198-f2c29537b4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:35.647070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb3e28-15e7-47a1-a198-f2c29537b4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:35.647462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb3e28-15e7-47a1-a198-f2c29537b4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:35.647741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb3e28-15e7-47a1-a198-f2c29537b4a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:35.648463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb3e28-15e7-47a1-a198-f2c29537b4a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:35.650357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb3e28-15e7-47a1-a198-f2c29537b4a2 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:35.680639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1103/2209] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:35 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:35.681732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984862c-02d9-4347-b5fa-41586eb982a0 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:35.692959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984862c-02d9-4347-b5fa-41586eb982a0 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:35.749957 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390c8d9-1a44-469d-ab47-e97ad09e373a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:35.750664 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390c8d9-1a44-469d-ab47-e97ad09e373a admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:35.750913 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390c8d9-1a44-469d-ab47-e97ad09e373a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:35.751192 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390c8d9-1a44-469d-ab47-e97ad09e373a admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:35.751902 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390c8d9-1a44-469d-ab47-e97ad09e373a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:35.753873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390c8d9-1a44-469d-ab47-e97ad09e373a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:35.775737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1107/2210] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:35 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:25:35.779756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb3e28-15e7-47a1-a198-f2c29537b4a2 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:35.787278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:35.791545 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb3e28-15e7-47a1-a198-f2c29537b4a2 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:35.797287 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe9e9a-9266-46ba-9f59-8549b89483a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:35.798336 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe9e9a-9266-46ba-9f59-8549b89483a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:35.798702 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe9e9a-9266-46ba-9f59-8549b89483a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:35.799410 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe9e9a-9266-46ba-9f59-8549b89483a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:35.800477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe9e9a-9266-46ba-9f59-8549b89483a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:35.802509 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe9e9a-9266-46ba-9f59-8549b89483a0 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:35.817033 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1104/2211] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:25:35.818713 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390c8d9-1a44-469d-ab47-e97ad09e373a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:35.825848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:35.830239 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390c8d9-1a44-469d-ab47-e97ad09e373a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:35.840158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20365329-21fc-4cc9-a637-09ae7d4c019e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:35.841306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20365329-21fc-4cc9-a637-09ae7d4c019e admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:35.841889 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20365329-21fc-4cc9-a637-09ae7d4c019e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:35.842316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20365329-21fc-4cc9-a637-09ae7d4c019e admin admin] PATH_INFO: `/v3/projects/7ac33d44ac7f40c7ad4c4efe4dc7a530/users/d49370e81db14621ac897d3a8b51eb71/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:35.866531 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20365329-21fc-4cc9-a637-09ae7d4c019e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac33d44ac7f40c7ad4c4efe4dc7a530, user_id=d49370e81db14621ac897d3a8b51eb71, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:35.869146 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20365329-21fc-4cc9-a637-09ae7d4c019e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:35.904366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1108/2212] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:35 2022] PUT /identity/v3/projects/7ac33d44ac7f40c7ad4c4efe4dc7a530/users/d49370e81db14621ac897d3a8b51eb71/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:35.907304 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe9e9a-9266-46ba-9f59-8549b89483a0 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:35.918697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe9e9a-9266-46ba-9f59-8549b89483a0 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:35.977544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566295cb-5bfc-484f-87b0-5ee445d9107a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:35.978211 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566295cb-5bfc-484f-87b0-5ee445d9107a admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:35.978497 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566295cb-5bfc-484f-87b0-5ee445d9107a admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:35.978782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566295cb-5bfc-484f-87b0-5ee445d9107a admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:35.979510 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566295cb-5bfc-484f-87b0-5ee445d9107a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:35.981126 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566295cb-5bfc-484f-87b0-5ee445d9107a admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:35.990597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1105/2213] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:35.994738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20365329-21fc-4cc9-a637-09ae7d4c019e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:36.004884 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20365329-21fc-4cc9-a637-09ae7d4c019e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:36.009948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2901c45-b041-4838-af98-3691837a3e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:36.010809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2901c45-b041-4838-af98-3691837a3e9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:36.011084 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2901c45-b041-4838-af98-3691837a3e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:36.011353 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2901c45-b041-4838-af98-3691837a3e9d admin admin] PATH_INFO: `/v3/projects/7ac33d44ac7f40c7ad4c4efe4dc7a530/users/d49370e81db14621ac897d3a8b51eb71/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:36.013347 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2901c45-b041-4838-af98-3691837a3e9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac33d44ac7f40c7ad4c4efe4dc7a530, user_id=d49370e81db14621ac897d3a8b51eb71, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:36.015942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2901c45-b041-4838-af98-3691837a3e9d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:36.026914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1109/2214] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:35 2022] PUT /identity/v3/projects/7ac33d44ac7f40c7ad4c4efe4dc7a530/users/d49370e81db14621ac897d3a8b51eb71/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:36.624092 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f1240-e2a2-4a54-ab2a-af8c8dc67963 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:36.624412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f1240-e2a2-4a54-ab2a-af8c8dc67963 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:36.624692 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f1240-e2a2-4a54-ab2a-af8c8dc67963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:36.651151 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-f26f1240-e2a2-4a54-ab2a-af8c8dc67963 None None] MFA Rules not processed for user `d49370e81db14621ac897d3a8b51eb71`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:36.653084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f26f1240-e2a2-4a54-ab2a-af8c8dc67963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:36.723887 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1106/2215] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:36.735829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2901c45-b041-4838-af98-3691837a3e9d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:36.737634 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b2901c45-b041-4838-af98-3691837a3e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:36.796092 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2901c45-b041-4838-af98-3691837a3e9d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:36.799758 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcc785-1e57-474c-8134-b5949a9cb89b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:36.800436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcc785-1e57-474c-8134-b5949a9cb89b service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:36.800721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcc785-1e57-474c-8134-b5949a9cb89b service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:36.800997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcc785-1e57-474c-8134-b5949a9cb89b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:36.856469 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bcc785-1e57-474c-8134-b5949a9cb89b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:36.857897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bcc785-1e57-474c-8134-b5949a9cb89b service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:36.868143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1110/2216] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:25:38.339991 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f1240-e2a2-4a54-ab2a-af8c8dc67963 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:38.361940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f1240-e2a2-4a54-ab2a-af8c8dc67963 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:38.369156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa62fdb-0c2a-4632-92c7-bd6215890f4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:38.370116 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa62fdb-0c2a-4632-92c7-bd6215890f4b service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:38.370677 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa62fdb-0c2a-4632-92c7-bd6215890f4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:38.371111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa62fdb-0c2a-4632-92c7-bd6215890f4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:38.384735 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa62fdb-0c2a-4632-92c7-bd6215890f4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:38.387334 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa62fdb-0c2a-4632-92c7-bd6215890f4b service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:38.413172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1107/2217] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:38 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:39.480812 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a40747-61e7-4929-add3-ec04b3b33050 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:39.481693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a40747-61e7-4929-add3-ec04b3b33050 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:39.482183 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a40747-61e7-4929-add3-ec04b3b33050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:39.504614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-90a40747-61e7-4929-add3-ec04b3b33050 None None] MFA Rules not processed for user `4a4f4d585adf40b880be44c29ecd53cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:39.507647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-90a40747-61e7-4929-add3-ec04b3b33050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:39.594762 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1111/2218] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:41.016435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa62fdb-0c2a-4632-92c7-bd6215890f4b service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:41.017867 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6aa62fdb-0c2a-4632-92c7-bd6215890f4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:41.086369 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa62fdb-0c2a-4632-92c7-bd6215890f4b service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:41.091093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576590e7-ba27-4ba3-8f6c-dad7a24eab55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:41.091983 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576590e7-ba27-4ba3-8f6c-dad7a24eab55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:41.092240 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576590e7-ba27-4ba3-8f6c-dad7a24eab55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:41.092562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576590e7-ba27-4ba3-8f6c-dad7a24eab55 admin admin] PATH_INFO: `/v3/users/9a1c8ca8857844919445d265b64d2192` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:41.093755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576590e7-ba27-4ba3-8f6c-dad7a24eab55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a1c8ca8857844919445d265b64d2192)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:41.095953 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576590e7-ba27-4ba3-8f6c-dad7a24eab55 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:41.098815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:41.164596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-576590e7-ba27-4ba3-8f6c-dad7a24eab55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a1c8ca8857844919445d265b64d2192'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:41.173665 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-576590e7-ba27-4ba3-8f6c-dad7a24eab55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a1c8ca8857844919445d265b64d2192'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:41.193441 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-576590e7-ba27-4ba3-8f6c-dad7a24eab55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a1c8ca8857844919445d265b64d2192'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:41.202145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-576590e7-ba27-4ba3-8f6c-dad7a24eab55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a1c8ca8857844919445d265b64d2192'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:41.203222 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-576590e7-ba27-4ba3-8f6c-dad7a24eab55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a1c8ca8857844919445d265b64d2192'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:41.213081 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1108/2219] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:41 2022] DELETE /identity/v3/users/9a1c8ca8857844919445d265b64d2192 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:41.217531 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9abcd2-22ae-4291-bd59-a66e01df9eb0 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:41.218497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9abcd2-22ae-4291-bd59-a66e01df9eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:41.219051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9abcd2-22ae-4291-bd59-a66e01df9eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:41.240526 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3a9abcd2-22ae-4291-bd59-a66e01df9eb0 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:41.242646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3a9abcd2-22ae-4291-bd59-a66e01df9eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:41.270787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:41.320817 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1112/2220] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:41.332987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576590e7-ba27-4ba3-8f6c-dad7a24eab55 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:41.343387 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-576590e7-ba27-4ba3-8f6c-dad7a24eab55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:41.414426 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576590e7-ba27-4ba3-8f6c-dad7a24eab55 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:41.418371 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27b580-e14e-4d3d-8caf-f7f423e4fb17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:41.419029 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27b580-e14e-4d3d-8caf-f7f423e4fb17 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:41.419346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27b580-e14e-4d3d-8caf-f7f423e4fb17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:41.419625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27b580-e14e-4d3d-8caf-f7f423e4fb17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:41.431081 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27b580-e14e-4d3d-8caf-f7f423e4fb17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:41.432538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27b580-e14e-4d3d-8caf-f7f423e4fb17 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:41.443444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1109/2221] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:41.797773 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9abcd2-22ae-4291-bd59-a66e01df9eb0 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:41.799385 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3a9abcd2-22ae-4291-bd59-a66e01df9eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:41.813891 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9abcd2-22ae-4291-bd59-a66e01df9eb0 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:41.819733 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ef5fa-6523-4364-9afc-ccf35b301936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:41.820740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ef5fa-6523-4364-9afc-ccf35b301936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:41.821233 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ef5fa-6523-4364-9afc-ccf35b301936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:41.821570 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ef5fa-6523-4364-9afc-ccf35b301936 admin admin] PATH_INFO: `/v3/projects/fac1b4711d514d53b1c08c841ee0af2f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:41.822856 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ef5fa-6523-4364-9afc-ccf35b301936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fac1b4711d514d53b1c08c841ee0af2f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:41.824893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ef5fa-6523-4364-9afc-ccf35b301936 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:41.874956 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8a0ef5fa-6523-4364-9afc-ccf35b301936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fac1b4711d514d53b1c08c841ee0af2f', 'request_id': 'req-8a0ef5fa-6523-4364-9afc-ccf35b301936'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:41.883750 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8a0ef5fa-6523-4364-9afc-ccf35b301936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fac1b4711d514d53b1c08c841ee0af2f', 'request_id': 'req-8a0ef5fa-6523-4364-9afc-ccf35b301936'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:41.894316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8a0ef5fa-6523-4364-9afc-ccf35b301936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fac1b4711d514d53b1c08c841ee0af2f', 'request_id': 'req-8a0ef5fa-6523-4364-9afc-ccf35b301936'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:41.908237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8a0ef5fa-6523-4364-9afc-ccf35b301936 admin admin] The token cache is being invalidate because project fac1b4711d514d53b1c08c841ee0af2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:41.908567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8a0ef5fa-6523-4364-9afc-ccf35b301936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:41.909226 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8a0ef5fa-6523-4364-9afc-ccf35b301936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:41.911357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1113/2222] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:41 2022] DELETE /identity/v3/projects/fac1b4711d514d53b1c08c841ee0af2f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:41.916446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da2a3-312f-46ca-bd23-61c3a46bfe7f None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:41.916833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da2a3-312f-46ca-bd23-61c3a46bfe7f None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:41.917124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da2a3-312f-46ca-bd23-61c3a46bfe7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:41.943439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-149da2a3-312f-46ca-bd23-61c3a46bfe7f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:41.946715 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-149da2a3-312f-46ca-bd23-61c3a46bfe7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:42.025185 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1110/2223] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:42.032568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc457d01-b203-437a-80b8-ef40cc3b3dee None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:42.033096 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc457d01-b203-437a-80b8-ef40cc3b3dee None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:42.033599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc457d01-b203-437a-80b8-ef40cc3b3dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:42.060010 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-fc457d01-b203-437a-80b8-ef40cc3b3dee None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:42.061318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fc457d01-b203-437a-80b8-ef40cc3b3dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:42.071540 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1114/2224] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:42.075907 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149da2a3-312f-46ca-bd23-61c3a46bfe7f None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:42.087422 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149da2a3-312f-46ca-bd23-61c3a46bfe7f None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:42.094760 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85925807-7898-4615-a5ac-2dc5acc728dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:42.095789 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85925807-7898-4615-a5ac-2dc5acc728dc admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:42.096097 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85925807-7898-4615-a5ac-2dc5acc728dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:42.096383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85925807-7898-4615-a5ac-2dc5acc728dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:42.097216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85925807-7898-4615-a5ac-2dc5acc728dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:42.097606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85925807-7898-4615-a5ac-2dc5acc728dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:42.099514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85925807-7898-4615-a5ac-2dc5acc728dc admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:42.117111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1111/2225] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:25:42.118142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc457d01-b203-437a-80b8-ef40cc3b3dee None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:42.128439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc457d01-b203-437a-80b8-ef40cc3b3dee None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:42.133918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4171ea6-2c95-455b-913e-53307ade11b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:42.134585 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4171ea6-2c95-455b-913e-53307ade11b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:42.135088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4171ea6-2c95-455b-913e-53307ade11b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:42.135377 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4171ea6-2c95-455b-913e-53307ade11b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:42.136527 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4171ea6-2c95-455b-913e-53307ade11b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:42.139144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4171ea6-2c95-455b-913e-53307ade11b3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:42.174212 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1115/2226] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:42 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:25:42.175967 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85925807-7898-4615-a5ac-2dc5acc728dc admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:42.186694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85925807-7898-4615-a5ac-2dc5acc728dc admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:42.212707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:42.268322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da3e90-2790-41af-a4c0-18f46d36209d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:42.269219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da3e90-2790-41af-a4c0-18f46d36209d admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:42.269508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da3e90-2790-41af-a4c0-18f46d36209d admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:42.269788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da3e90-2790-41af-a4c0-18f46d36209d admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:42.270490 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da3e90-2790-41af-a4c0-18f46d36209d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:42.272634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da3e90-2790-41af-a4c0-18f46d36209d admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:42.296655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1112/2227] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:42 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:25:42.298170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4171ea6-2c95-455b-913e-53307ade11b3 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:42.309777 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4171ea6-2c95-455b-913e-53307ade11b3 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:42.315946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f53a63-3a29-4a33-9c4c-10df0daa6a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:42.316861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f53a63-3a29-4a33-9c4c-10df0daa6a71 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:42.317377 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f53a63-3a29-4a33-9c4c-10df0daa6a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:42.317823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f53a63-3a29-4a33-9c4c-10df0daa6a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:42.319323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f53a63-3a29-4a33-9c4c-10df0daa6a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:42.321170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f53a63-3a29-4a33-9c4c-10df0daa6a71 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:42.341490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1116/2228] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:42.342178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da3e90-2790-41af-a4c0-18f46d36209d admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:42.357333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da3e90-2790-41af-a4c0-18f46d36209d admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:42.364223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50246bb1-19db-4eb0-9bd4-fdd5f2a54154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:42.365510 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50246bb1-19db-4eb0-9bd4-fdd5f2a54154 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:42.365798 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50246bb1-19db-4eb0-9bd4-fdd5f2a54154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:42.366080 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50246bb1-19db-4eb0-9bd4-fdd5f2a54154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:42.366879 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50246bb1-19db-4eb0-9bd4-fdd5f2a54154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:42.368542 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50246bb1-19db-4eb0-9bd4-fdd5f2a54154 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:42.379857 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1113/2229] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:42.384441 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f53a63-3a29-4a33-9c4c-10df0daa6a71 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:42.395997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f53a63-3a29-4a33-9c4c-10df0daa6a71 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:42.401344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbf3ef-3b36-42b8-95e4-a24540636b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:42.402251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbf3ef-3b36-42b8-95e4-a24540636b3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:42.402537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbf3ef-3b36-42b8-95e4-a24540636b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:42.402815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbf3ef-3b36-42b8-95e4-a24540636b3f admin admin] PATH_INFO: `/v3/projects/78716b9626bb49bc9c55f53bf9a0657d/users/12d595827a894a81ba21083372352982/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:42.419425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbf3ef-3b36-42b8-95e4-a24540636b3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=78716b9626bb49bc9c55f53bf9a0657d, user_id=12d595827a894a81ba21083372352982, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:42.422273 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbf3ef-3b36-42b8-95e4-a24540636b3f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:42.452040 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1117/2230] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:42 2022] PUT /identity/v3/projects/78716b9626bb49bc9c55f53bf9a0657d/users/12d595827a894a81ba21083372352982/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:42.463803 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05757ba-8840-46a0-becd-3ede19c9a871 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:42.464081 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05757ba-8840-46a0-becd-3ede19c9a871 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:42.464327 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05757ba-8840-46a0-becd-3ede19c9a871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:42.500792 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-d05757ba-8840-46a0-becd-3ede19c9a871 None None] MFA Rules not processed for user `12d595827a894a81ba21083372352982`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:42.502111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d05757ba-8840-46a0-becd-3ede19c9a871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:42.594259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1114/2231] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:42.616064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbf3ef-3b36-42b8-95e4-a24540636b3f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:42.619441 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-90cbf3ef-3b36-42b8-95e4-a24540636b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:42.772895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:42.792957 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbf3ef-3b36-42b8-95e4-a24540636b3f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:42.809985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f595032-e380-4324-9d40-0c9578fb9d31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:42.812941 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f595032-e380-4324-9d40-0c9578fb9d31 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:42.813593 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f595032-e380-4324-9d40-0c9578fb9d31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:42.814621 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f595032-e380-4324-9d40-0c9578fb9d31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:42.843101 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f595032-e380-4324-9d40-0c9578fb9d31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:42.851264 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f595032-e380-4324-9d40-0c9578fb9d31 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:42.896545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1118/2232] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:52.588695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=61,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:52.867789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:58.377048 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05757ba-8840-46a0-becd-3ede19c9a871 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:58.378805 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d05757ba-8840-46a0-becd-3ede19c9a871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:58.449506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05757ba-8840-46a0-becd-3ede19c9a871 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:58.454123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4401a99-e649-4405-aae2-3182028ec8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:58.455004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4401a99-e649-4405-aae2-3182028ec8ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:58.455290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4401a99-e649-4405-aae2-3182028ec8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:58.455567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4401a99-e649-4405-aae2-3182028ec8ce admin admin] PATH_INFO: `/v3/users/108789b84d30459ebc9b3e916f2f96a0` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:58.456846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4401a99-e649-4405-aae2-3182028ec8ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=108789b84d30459ebc9b3e916f2f96a0)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:58.459441 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4401a99-e649-4405-aae2-3182028ec8ce admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:58.509568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b4401a99-e649-4405-aae2-3182028ec8ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '108789b84d30459ebc9b3e916f2f96a0'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:58.516734 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b4401a99-e649-4405-aae2-3182028ec8ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '108789b84d30459ebc9b3e916f2f96a0'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:58.531009 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b4401a99-e649-4405-aae2-3182028ec8ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '108789b84d30459ebc9b3e916f2f96a0'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:58.539087 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b4401a99-e649-4405-aae2-3182028ec8ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '108789b84d30459ebc9b3e916f2f96a0'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:58.540090 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b4401a99-e649-4405-aae2-3182028ec8ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '108789b84d30459ebc9b3e916f2f96a0'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:58.548083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1115/2233] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:58 2022] DELETE /identity/v3/users/108789b84d30459ebc9b3e916f2f96a0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:58.726365 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f595032-e380-4324-9d40-0c9578fb9d31 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:58.739290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f595032-e380-4324-9d40-0c9578fb9d31 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:58.798022 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:58.798887 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:58.799172 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:58.799451 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1 admin admin] PATH_INFO: `/v3/projects/4f13b0a634b8423581a33819b24a1ffc` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:58.800702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f13b0a634b8423581a33819b24a1ffc)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:58.803146 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:58.849943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f13b0a634b8423581a33819b24a1ffc', 'request_id': 'req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:58.855908 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f13b0a634b8423581a33819b24a1ffc', 'request_id': 'req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:58.863714 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f13b0a634b8423581a33819b24a1ffc', 'request_id': 'req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:58.874501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1 admin admin] The token cache is being invalidate because project 4f13b0a634b8423581a33819b24a1ffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:58.874785 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:58.875412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:58.877006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1119/2234] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:58 2022] DELETE /identity/v3/projects/4f13b0a634b8423581a33819b24a1ffc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:58.881800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4401a99-e649-4405-aae2-3182028ec8ce admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:58.882943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-b4401a99-e649-4405-aae2-3182028ec8ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:58.895677 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4401a99-e649-4405-aae2-3182028ec8ce admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:58.952980 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65b690-bbe9-4f40-aaba-70fc01864c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:58.953869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65b690-bbe9-4f40-aaba-70fc01864c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:58.954155 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65b690-bbe9-4f40-aaba-70fc01864c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:58.954429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65b690-bbe9-4f40-aaba-70fc01864c64 admin admin] PATH_INFO: `/v3/users/d49370e81db14621ac897d3a8b51eb71` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:58.955614 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65b690-bbe9-4f40-aaba-70fc01864c64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d49370e81db14621ac897d3a8b51eb71)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:58.957556 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65b690-bbe9-4f40-aaba-70fc01864c64 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:58.979570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:59.005146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-bd65b690-bbe9-4f40-aaba-70fc01864c64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd49370e81db14621ac897d3a8b51eb71'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:59.014244 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-bd65b690-bbe9-4f40-aaba-70fc01864c64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd49370e81db14621ac897d3a8b51eb71'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:59.031731 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-bd65b690-bbe9-4f40-aaba-70fc01864c64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd49370e81db14621ac897d3a8b51eb71'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:59.038645 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-bd65b690-bbe9-4f40-aaba-70fc01864c64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd49370e81db14621ac897d3a8b51eb71'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:59.039359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-bd65b690-bbe9-4f40-aaba-70fc01864c64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd49370e81db14621ac897d3a8b51eb71'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:59.045344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1116/2235] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:58 2022] DELETE /identity/v3/users/d49370e81db14621ac897d3a8b51eb71 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:59.239355 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:59.241479 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:59.259516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb8454-2a21-4dbb-a59e-dc7b14b785d1 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:59.322755 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed419004-0726-482c-a2ba-93b9ca212a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:59.323653 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed419004-0726-482c-a2ba-93b9ca212a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:59.323948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed419004-0726-482c-a2ba-93b9ca212a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:59.324204 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed419004-0726-482c-a2ba-93b9ca212a4e admin admin] PATH_INFO: `/v3/projects/7ac33d44ac7f40c7ad4c4efe4dc7a530` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:59.325601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed419004-0726-482c-a2ba-93b9ca212a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ac33d44ac7f40c7ad4c4efe4dc7a530)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:59.327516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed419004-0726-482c-a2ba-93b9ca212a4e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:59.372490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:59.388553 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ed419004-0726-482c-a2ba-93b9ca212a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ac33d44ac7f40c7ad4c4efe4dc7a530', 'request_id': 'req-ed419004-0726-482c-a2ba-93b9ca212a4e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:59.395501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ed419004-0726-482c-a2ba-93b9ca212a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ac33d44ac7f40c7ad4c4efe4dc7a530', 'request_id': 'req-ed419004-0726-482c-a2ba-93b9ca212a4e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:59.406336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ed419004-0726-482c-a2ba-93b9ca212a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ac33d44ac7f40c7ad4c4efe4dc7a530', 'request_id': 'req-ed419004-0726-482c-a2ba-93b9ca212a4e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:59.419363 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ed419004-0726-482c-a2ba-93b9ca212a4e admin admin] The token cache is being invalidate because project 7ac33d44ac7f40c7ad4c4efe4dc7a530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:59.419697 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ed419004-0726-482c-a2ba-93b9ca212a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:59.420339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ed419004-0726-482c-a2ba-93b9ca212a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:59.422077 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1120/2236] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:59 2022] DELETE /identity/v3/projects/7ac33d44ac7f40c7ad4c4efe4dc7a530 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:59.426834 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573e37a-4fcc-46c9-a04e-cbbf5dc67ef5 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:59.428382 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573e37a-4fcc-46c9-a04e-cbbf5dc67ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:59.428950 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573e37a-4fcc-46c9-a04e-cbbf5dc67ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:59.448308 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-8573e37a-4fcc-46c9-a04e-cbbf5dc67ef5 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:59.449916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-8573e37a-4fcc-46c9-a04e-cbbf5dc67ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:59.525177 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1117/2237] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:59.533376 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d93c0-9724-40f7-8da6-69c3bec6d367 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:59.533926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d93c0-9724-40f7-8da6-69c3bec6d367 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:59.534334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d93c0-9724-40f7-8da6-69c3bec6d367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:59.554000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c59d93c0-9724-40f7-8da6-69c3bec6d367 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:59.555325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c59d93c0-9724-40f7-8da6-69c3bec6d367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:59.563621 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1121/2238] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:59.567951 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573e37a-4fcc-46c9-a04e-cbbf5dc67ef5 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:59.578209 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573e37a-4fcc-46c9-a04e-cbbf5dc67ef5 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:59.582994 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b6d81-f1e7-4ac9-a190-81076837f3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:59.584157 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b6d81-f1e7-4ac9-a190-81076837f3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:59.584468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b6d81-f1e7-4ac9-a190-81076837f3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:59.584747 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b6d81-f1e7-4ac9-a190-81076837f3b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:59.585492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b6d81-f1e7-4ac9-a190-81076837f3b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:59.585801 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b6d81-f1e7-4ac9-a190-81076837f3b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:59.587379 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b6d81-f1e7-4ac9-a190-81076837f3b3 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:59.599921 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1118/2239] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:25:59.604367 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d93c0-9724-40f7-8da6-69c3bec6d367 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:59.614417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d93c0-9724-40f7-8da6-69c3bec6d367 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:59.619103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c5fa0-edef-454b-8570-109c7cc9770f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:59.619757 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c5fa0-edef-454b-8570-109c7cc9770f admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:59.620036 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c5fa0-edef-454b-8570-109c7cc9770f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:59.620332 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c5fa0-edef-454b-8570-109c7cc9770f admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:59.621048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c5fa0-edef-454b-8570-109c7cc9770f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:59.622945 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c5fa0-edef-454b-8570-109c7cc9770f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:59.651939 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1122/2240] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:59 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:59.656593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b6d81-f1e7-4ac9-a190-81076837f3b3 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:59.670736 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b6d81-f1e7-4ac9-a190-81076837f3b3 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:59.736263 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa6877-5121-4616-8f03-1c40ab76dbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:59.737235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa6877-5121-4616-8f03-1c40ab76dbaa admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:59.737521 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa6877-5121-4616-8f03-1c40ab76dbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:59.737802 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa6877-5121-4616-8f03-1c40ab76dbaa admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:59.738511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa6877-5121-4616-8f03-1c40ab76dbaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:59.740478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa6877-5121-4616-8f03-1c40ab76dbaa admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:59.761218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1119/2241] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:59 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:59.765285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c5fa0-edef-454b-8570-109c7cc9770f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:59.776724 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c5fa0-edef-454b-8570-109c7cc9770f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:59.781400 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a32f3-fabc-4787-85d9-1c171c70bfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:59.786122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a32f3-fabc-4787-85d9-1c171c70bfbb admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:59.786122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a32f3-fabc-4787-85d9-1c171c70bfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:59.786122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a32f3-fabc-4787-85d9-1c171c70bfbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:59.786122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a32f3-fabc-4787-85d9-1c171c70bfbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:59.786122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a32f3-fabc-4787-85d9-1c171c70bfbb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:59.796377 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1123/2242] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:25:59.800261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa6877-5121-4616-8f03-1c40ab76dbaa admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:59.810316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa6877-5121-4616-8f03-1c40ab76dbaa admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:59.815203 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041751ec-368f-46c9-8982-159bab30deaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:59.815865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041751ec-368f-46c9-8982-159bab30deaf admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:59.816150 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041751ec-368f-46c9-8982-159bab30deaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:59.816470 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041751ec-368f-46c9-8982-159bab30deaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:59.817195 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041751ec-368f-46c9-8982-159bab30deaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:59.819293 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041751ec-368f-46c9-8982-159bab30deaf admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:59.829509 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1120/2243] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:59.834474 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a32f3-fabc-4787-85d9-1c171c70bfbb admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:59.844482 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a32f3-fabc-4787-85d9-1c171c70bfbb admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:59.849781 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658322f7-d27b-4c55-a830-1f87ab5b5698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:59.850791 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658322f7-d27b-4c55-a830-1f87ab5b5698 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:59.851079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658322f7-d27b-4c55-a830-1f87ab5b5698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:59.851357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658322f7-d27b-4c55-a830-1f87ab5b5698 admin admin] PATH_INFO: `/v3/projects/00b09a6b498b4ce89e3e31cc841aa945/users/5d4f3f34b63942f88b970f2fa06dc258/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:59.868516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658322f7-d27b-4c55-a830-1f87ab5b5698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00b09a6b498b4ce89e3e31cc841aa945, user_id=5d4f3f34b63942f88b970f2fa06dc258, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:59.871209 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658322f7-d27b-4c55-a830-1f87ab5b5698 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:59.885748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1124/2244] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:59 2022] PUT /identity/v3/projects/00b09a6b498b4ce89e3e31cc841aa945/users/5d4f3f34b63942f88b970f2fa06dc258/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:59.896701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60d5e4-d7fa-40c9-9ee5-bdc874bfbf8e None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:59.896993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60d5e4-d7fa-40c9-9ee5-bdc874bfbf8e None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:59.897269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60d5e4-d7fa-40c9-9ee5-bdc874bfbf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:59.905127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:59.929042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-8c60d5e4-d7fa-40c9-9ee5-bdc874bfbf8e None None] MFA Rules not processed for user `5d4f3f34b63942f88b970f2fa06dc258`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:59.930317 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-8c60d5e4-d7fa-40c9-9ee5-bdc874bfbf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:59.997105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1121/2245] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:25:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:00.001741 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658322f7-d27b-4c55-a830-1f87ab5b5698 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:00.066272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658322f7-d27b-4c55-a830-1f87ab5b5698 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:00.072561 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de21420-38a6-4d02-bee4-e67a276cbe30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:00.073211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de21420-38a6-4d02-bee4-e67a276cbe30 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:00.073492 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de21420-38a6-4d02-bee4-e67a276cbe30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:00.073768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de21420-38a6-4d02-bee4-e67a276cbe30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:00.074472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de21420-38a6-4d02-bee4-e67a276cbe30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:00.076382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de21420-38a6-4d02-bee4-e67a276cbe30 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:00.101910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1125/2246] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:26:00 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:26:00.105595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60d5e4-d7fa-40c9-9ee5-bdc874bfbf8e None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:00.121830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60d5e4-d7fa-40c9-9ee5-bdc874bfbf8e None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:00.197343 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4643027-a2a6-4951-b16d-7857c1ae9469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:00.198049 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4643027-a2a6-4951-b16d-7857c1ae9469 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:00.198329 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4643027-a2a6-4951-b16d-7857c1ae9469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:00.198614 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4643027-a2a6-4951-b16d-7857c1ae9469 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:00.199318 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4643027-a2a6-4951-b16d-7857c1ae9469 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:00.201284 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4643027-a2a6-4951-b16d-7857c1ae9469 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:00.224473 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1122/2247] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:26:00 2022] POST /identity/v3/users => generated 298 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:26:00.228258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de21420-38a6-4d02-bee4-e67a276cbe30 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:00.239402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de21420-38a6-4d02-bee4-e67a276cbe30 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:00.244700 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e09f74-6424-4377-b25f-7ad4d7e0253e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:00.245369 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e09f74-6424-4377-b25f-7ad4d7e0253e admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:00.245657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e09f74-6424-4377-b25f-7ad4d7e0253e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:00.245935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e09f74-6424-4377-b25f-7ad4d7e0253e admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:00.246668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e09f74-6424-4377-b25f-7ad4d7e0253e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:00.248352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e09f74-6424-4377-b25f-7ad4d7e0253e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:00.262954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1126/2248] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:26:00.264527 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4643027-a2a6-4951-b16d-7857c1ae9469 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:00.274872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4643027-a2a6-4951-b16d-7857c1ae9469 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:00.279543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb1815-52ad-4816-94e5-d04f8327205c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:00.280466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb1815-52ad-4816-94e5-d04f8327205c admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:00.280749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb1815-52ad-4816-94e5-d04f8327205c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:00.281025 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb1815-52ad-4816-94e5-d04f8327205c admin admin] PATH_INFO: `/v3/projects/c92db13a24b149a78d254a042b4a3b36/users/1f9a65c34b0e4ecd8a0f5d5db62808df/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:00.295591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb1815-52ad-4816-94e5-d04f8327205c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c92db13a24b149a78d254a042b4a3b36, user_id=1f9a65c34b0e4ecd8a0f5d5db62808df, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:00.298275 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb1815-52ad-4816-94e5-d04f8327205c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:00.311268 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1123/2249] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:26:00 2022] PUT /identity/v3/projects/c92db13a24b149a78d254a042b4a3b36/users/1f9a65c34b0e4ecd8a0f5d5db62808df/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:00.315453 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e09f74-6424-4377-b25f-7ad4d7e0253e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:00.326093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e09f74-6424-4377-b25f-7ad4d7e0253e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:00.358555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:00.395525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae04086-eea5-49e7-9a16-449979cc364a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:00.396186 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae04086-eea5-49e7-9a16-449979cc364a admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:00.396515 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae04086-eea5-49e7-9a16-449979cc364a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:00.396771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae04086-eea5-49e7-9a16-449979cc364a admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:00.397493 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae04086-eea5-49e7-9a16-449979cc364a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:00.399073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae04086-eea5-49e7-9a16-449979cc364a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:00.408723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1127/2250] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:26:00.414759 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb1815-52ad-4816-94e5-d04f8327205c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:00.425099 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb1815-52ad-4816-94e5-d04f8327205c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:00.430244 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835e900-0e4e-4bf8-a906-f4fe1ca7fed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:00.431118 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835e900-0e4e-4bf8-a906-f4fe1ca7fed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:00.431405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835e900-0e4e-4bf8-a906-f4fe1ca7fed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:00.431682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835e900-0e4e-4bf8-a906-f4fe1ca7fed5 admin admin] PATH_INFO: `/v3/projects/c92db13a24b149a78d254a042b4a3b36/users/1f9a65c34b0e4ecd8a0f5d5db62808df/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:00.433708 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835e900-0e4e-4bf8-a906-f4fe1ca7fed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c92db13a24b149a78d254a042b4a3b36, user_id=1f9a65c34b0e4ecd8a0f5d5db62808df, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:00.436378 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835e900-0e4e-4bf8-a906-f4fe1ca7fed5 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:00.448707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1124/2251] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:26:00 2022] PUT /identity/v3/projects/c92db13a24b149a78d254a042b4a3b36/users/1f9a65c34b0e4ecd8a0f5d5db62808df/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:00.459705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd4a6e-4792-4498-9526-25c150af9df5 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:00.460000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd4a6e-4792-4498-9526-25c150af9df5 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:00.460279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd4a6e-4792-4498-9526-25c150af9df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:00.487289 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-58dd4a6e-4792-4498-9526-25c150af9df5 None None] MFA Rules not processed for user `1f9a65c34b0e4ecd8a0f5d5db62808df`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:00.488542 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-58dd4a6e-4792-4498-9526-25c150af9df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:00.567515 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1128/2252] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:00.578923 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835e900-0e4e-4bf8-a906-f4fe1ca7fed5 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:00.580712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-9835e900-0e4e-4bf8-a906-f4fe1ca7fed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:00.648645 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:00.656108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835e900-0e4e-4bf8-a906-f4fe1ca7fed5 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:00.662009 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f85469-0c10-4273-bc08-5ec5c80375e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:00.662762 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f85469-0c10-4273-bc08-5ec5c80375e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:00.663036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f85469-0c10-4273-bc08-5ec5c80375e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:00.663683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f85469-0c10-4273-bc08-5ec5c80375e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:00.727235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f85469-0c10-4273-bc08-5ec5c80375e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:00.728704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f85469-0c10-4273-bc08-5ec5c80375e8 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:00.738492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1125/2253] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:03.432309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd4a6e-4792-4498-9526-25c150af9df5 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:03.433851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-58dd4a6e-4792-4498-9526-25c150af9df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:03.516676 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd4a6e-4792-4498-9526-25c150af9df5 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:03.530035 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be99c2-94a0-4346-874e-9ac592e8508e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:03.530955 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be99c2-94a0-4346-874e-9ac592e8508e service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:03.531348 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be99c2-94a0-4346-874e-9ac592e8508e service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:03.531633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be99c2-94a0-4346-874e-9ac592e8508e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:03.534488 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c7be99c2-94a0-4346-874e-9ac592e8508e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:03.624517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be99c2-94a0-4346-874e-9ac592e8508e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:03.628497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be99c2-94a0-4346-874e-9ac592e8508e service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:03.639460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1129/2254] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:08.301143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f85469-0c10-4273-bc08-5ec5c80375e8 service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:08.302581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-44f85469-0c10-4273-bc08-5ec5c80375e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:08.313967 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f85469-0c10-4273-bc08-5ec5c80375e8 service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:08.318310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e218f-0e44-44cf-953a-7c8e92983dee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:08.318949 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e218f-0e44-44cf-953a-7c8e92983dee service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:08.319235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e218f-0e44-44cf-953a-7c8e92983dee service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:08.319510 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e218f-0e44-44cf-953a-7c8e92983dee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:08.329820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e218f-0e44-44cf-953a-7c8e92983dee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:08.331282 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e218f-0e44-44cf-953a-7c8e92983dee service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:08.344133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1126/2255] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:08.447560 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166116c-0fb5-4e27-b9db-fbd904c40551 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:08.448035 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166116c-0fb5-4e27-b9db-fbd904c40551 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:08.448467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166116c-0fb5-4e27-b9db-fbd904c40551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:08.456222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:08.471080 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1166116c-0fb5-4e27-b9db-fbd904c40551 None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:08.472535 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1166116c-0fb5-4e27-b9db-fbd904c40551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:08.494042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e218f-0e44-44cf-953a-7c8e92983dee service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:08.497417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f20e218f-0e44-44cf-953a-7c8e92983dee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:08.539319 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1130/2256] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jul 20 19:26:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:08.598668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166116c-0fb5-4e27-b9db-fbd904c40551 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:08.600829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1166116c-0fb5-4e27-b9db-fbd904c40551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:08.609027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e218f-0e44-44cf-953a-7c8e92983dee service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:08.614384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af29ef8-f198-4a42-b619-85177941b440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:08.615802 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af29ef8-f198-4a42-b619-85177941b440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:08.616085 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af29ef8-f198-4a42-b619-85177941b440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:08.616896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af29ef8-f198-4a42-b619-85177941b440 admin admin] PATH_INFO: `/v3/users/12d595827a894a81ba21083372352982` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:08.619648 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af29ef8-f198-4a42-b619-85177941b440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12d595827a894a81ba21083372352982)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:08.621872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af29ef8-f198-4a42-b619-85177941b440 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:08.691288 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-7af29ef8-f198-4a42-b619-85177941b440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12d595827a894a81ba21083372352982'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:08.700608 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-7af29ef8-f198-4a42-b619-85177941b440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12d595827a894a81ba21083372352982'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:08.723398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-7af29ef8-f198-4a42-b619-85177941b440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12d595827a894a81ba21083372352982'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:08.733935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-7af29ef8-f198-4a42-b619-85177941b440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12d595827a894a81ba21083372352982'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:08.734781 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-7af29ef8-f198-4a42-b619-85177941b440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12d595827a894a81ba21083372352982'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:08.743597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1127/2257] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:26:08 2022] DELETE /identity/v3/users/12d595827a894a81ba21083372352982 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:26:08.750914 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513f4d6-f593-42ea-b8ba-c2a3d76e1b88 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:08.751314 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513f4d6-f593-42ea-b8ba-c2a3d76e1b88 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:08.753556 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513f4d6-f593-42ea-b8ba-c2a3d76e1b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:08.761616 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166116c-0fb5-4e27-b9db-fbd904c40551 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:08.777861 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:08.806399 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4513f4d6-f593-42ea-b8ba-c2a3d76e1b88 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:08.807845 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4513f4d6-f593-42ea-b8ba-c2a3d76e1b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:08.888879 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271147c2-fbf7-40a9-a053-bb2d8c90f169 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:08.889636 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271147c2-fbf7-40a9-a053-bb2d8c90f169 service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:08.889926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271147c2-fbf7-40a9-a053-bb2d8c90f169 service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:08.890200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271147c2-fbf7-40a9-a053-bb2d8c90f169 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:08.892871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-271147c2-fbf7-40a9-a053-bb2d8c90f169 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:08.897583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1128/2258] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:08.909057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513f4d6-f593-42ea-b8ba-c2a3d76e1b88 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:08.910900 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4513f4d6-f593-42ea-b8ba-c2a3d76e1b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:08.949625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:08.968895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271147c2-fbf7-40a9-a053-bb2d8c90f169 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:08.970699 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271147c2-fbf7-40a9-a053-bb2d8c90f169 service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:08.984174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1131/2259] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:26:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 19:26:08.999975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513f4d6-f593-42ea-b8ba-c2a3d76e1b88 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:09.004080 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617a420-fe61-4c68-bc0a-0228f642ab42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:09.004780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617a420-fe61-4c68-bc0a-0228f642ab42 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:09.005089 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617a420-fe61-4c68-bc0a-0228f642ab42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:09.005439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617a420-fe61-4c68-bc0a-0228f642ab42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:09.015850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617a420-fe61-4c68-bc0a-0228f642ab42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:09.017305 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617a420-fe61-4c68-bc0a-0228f642ab42 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:09.032925 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1129/2260] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:26:09.510577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271147c2-fbf7-40a9-a053-bb2d8c90f169 service swift] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:09.512018 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-271147c2-fbf7-40a9-a053-bb2d8c90f169 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:09.541714 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271147c2-fbf7-40a9-a053-bb2d8c90f169 service swift] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:09.558349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa682ea-7472-4bff-8414-c04d5489c327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:09.559525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa682ea-7472-4bff-8414-c04d5489c327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:09.560003 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa682ea-7472-4bff-8414-c04d5489c327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:09.560321 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa682ea-7472-4bff-8414-c04d5489c327 admin admin] PATH_INFO: `/v3/projects/78716b9626bb49bc9c55f53bf9a0657d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:09.562952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa682ea-7472-4bff-8414-c04d5489c327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78716b9626bb49bc9c55f53bf9a0657d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:09.565011 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa682ea-7472-4bff-8414-c04d5489c327 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:09.729213 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2fa682ea-7472-4bff-8414-c04d5489c327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78716b9626bb49bc9c55f53bf9a0657d', 'request_id': 'req-2fa682ea-7472-4bff-8414-c04d5489c327'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:09.738231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2fa682ea-7472-4bff-8414-c04d5489c327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78716b9626bb49bc9c55f53bf9a0657d', 'request_id': 'req-2fa682ea-7472-4bff-8414-c04d5489c327'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:09.750334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2fa682ea-7472-4bff-8414-c04d5489c327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78716b9626bb49bc9c55f53bf9a0657d', 'request_id': 'req-2fa682ea-7472-4bff-8414-c04d5489c327'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:09.767249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2fa682ea-7472-4bff-8414-c04d5489c327 admin admin] The token cache is being invalidate because project 78716b9626bb49bc9c55f53bf9a0657d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:26:09.767682 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2fa682ea-7472-4bff-8414-c04d5489c327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:09.769080 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2fa682ea-7472-4bff-8414-c04d5489c327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:09.775513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1132/2261] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:26:09 2022] DELETE /identity/v3/projects/78716b9626bb49bc9c55f53bf9a0657d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:09.777183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fe4c9-ad3d-45bb-8d95-12a80f5dc639 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:09.777695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fe4c9-ad3d-45bb-8d95-12a80f5dc639 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:09.777797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fe4c9-ad3d-45bb-8d95-12a80f5dc639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:09.829517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-610fe4c9-ad3d-45bb-8d95-12a80f5dc639 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:09.830843 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-610fe4c9-ad3d-45bb-8d95-12a80f5dc639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:09.974502 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1130/2262] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:26:09.977670 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fd099-7605-46c7-9290-479cb67d93be None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:09.978313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fd099-7605-46c7-9290-479cb67d93be None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:09.978802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fd099-7605-46c7-9290-479cb67d93be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:10.024560 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0a1fd099-7605-46c7-9290-479cb67d93be None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:10.030071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0a1fd099-7605-46c7-9290-479cb67d93be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:10.044683 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1133/2263] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:10.048448 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fe4c9-ad3d-45bb-8d95-12a80f5dc639 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:10.059892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fe4c9-ad3d-45bb-8d95-12a80f5dc639 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:10.070139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d99ac-ca95-424b-8204-a735b1625cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:10.070995 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d99ac-ca95-424b-8204-a735b1625cbb admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:10.071385 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d99ac-ca95-424b-8204-a735b1625cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:10.071761 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d99ac-ca95-424b-8204-a735b1625cbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:10.083163 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d99ac-ca95-424b-8204-a735b1625cbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:26:10.083163 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d99ac-ca95-424b-8204-a735b1625cbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:10.087736 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d99ac-ca95-424b-8204-a735b1625cbb admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:10.118698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fd099-7605-46c7-9290-479cb67d93be None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:10.124468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1131/2264] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:26:10.128750 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fd099-7605-46c7-9290-479cb67d93be None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:10.134022 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76404d62-1d47-4987-b989-60892db02a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:10.134851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76404d62-1d47-4987-b989-60892db02a8b admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:10.135208 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76404d62-1d47-4987-b989-60892db02a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:10.135574 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76404d62-1d47-4987-b989-60892db02a8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:10.136412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76404d62-1d47-4987-b989-60892db02a8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:10.138684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76404d62-1d47-4987-b989-60892db02a8b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:10.249613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1134/2265] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:26:10 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:26:10.253682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d99ac-ca95-424b-8204-a735b1625cbb admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:10.270919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d99ac-ca95-424b-8204-a735b1625cbb admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:10.328982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:10.386161 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762c061-9c9b-4941-a5ca-406bea4bdc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:10.386973 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762c061-9c9b-4941-a5ca-406bea4bdc04 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:10.387303 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762c061-9c9b-4941-a5ca-406bea4bdc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:10.387657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762c061-9c9b-4941-a5ca-406bea4bdc04 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:10.388701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a762c061-9c9b-4941-a5ca-406bea4bdc04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:10.391423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a762c061-9c9b-4941-a5ca-406bea4bdc04 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:10.430859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1132/2266] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:26:10 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:26:10.432056 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76404d62-1d47-4987-b989-60892db02a8b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:10.449399 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76404d62-1d47-4987-b989-60892db02a8b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:10.456630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe95ce8-69ff-4f93-a637-afacf300745d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:10.457479 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe95ce8-69ff-4f93-a637-afacf300745d admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:10.457772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe95ce8-69ff-4f93-a637-afacf300745d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:10.458046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe95ce8-69ff-4f93-a637-afacf300745d admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:10.458978 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe95ce8-69ff-4f93-a637-afacf300745d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:10.460983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe95ce8-69ff-4f93-a637-afacf300745d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:10.484759 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1135/2267] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:26:10.489162 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762c061-9c9b-4941-a5ca-406bea4bdc04 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:10.514514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762c061-9c9b-4941-a5ca-406bea4bdc04 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:10.524170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e6ed0-d4e3-4022-9018-8f24aaa0f833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:10.524966 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e6ed0-d4e3-4022-9018-8f24aaa0f833 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:10.525254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e6ed0-d4e3-4022-9018-8f24aaa0f833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:10.525525 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e6ed0-d4e3-4022-9018-8f24aaa0f833 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:10.526259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e6ed0-d4e3-4022-9018-8f24aaa0f833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:10.527928 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e6ed0-d4e3-4022-9018-8f24aaa0f833 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:10.539774 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1133/2268] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:26:10.544868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe95ce8-69ff-4f93-a637-afacf300745d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:10.558171 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe95ce8-69ff-4f93-a637-afacf300745d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:10.564085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d51565-c418-4459-a510-024fd5ccb8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:10.565662 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d51565-c418-4459-a510-024fd5ccb8ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:10.565932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d51565-c418-4459-a510-024fd5ccb8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:10.566272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d51565-c418-4459-a510-024fd5ccb8ce admin admin] PATH_INFO: `/v3/projects/dd21ead5b3d24e43bcd1ab7bbda9a7da/users/659f14cafb0848b8852fe4db9dd7a681/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:10.586163 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d51565-c418-4459-a510-024fd5ccb8ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd21ead5b3d24e43bcd1ab7bbda9a7da, user_id=659f14cafb0848b8852fe4db9dd7a681, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:10.588798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d51565-c418-4459-a510-024fd5ccb8ce admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:10.608382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1136/2269] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:26:10 2022] PUT /identity/v3/projects/dd21ead5b3d24e43bcd1ab7bbda9a7da/users/659f14cafb0848b8852fe4db9dd7a681/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:10.618902 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319d1eb-aa47-4967-8e92-9ea162a988c8 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:10.619219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319d1eb-aa47-4967-8e92-9ea162a988c8 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:10.619792 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319d1eb-aa47-4967-8e92-9ea162a988c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:10.649451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4319d1eb-aa47-4967-8e92-9ea162a988c8 None None] MFA Rules not processed for user `659f14cafb0848b8852fe4db9dd7a681`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:10.650716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4319d1eb-aa47-4967-8e92-9ea162a988c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:10.736590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1134/2270] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:10.752814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d51565-c418-4459-a510-024fd5ccb8ce admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:10.754937 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a4d51565-c418-4459-a510-024fd5ccb8ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:10.832206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d51565-c418-4459-a510-024fd5ccb8ce admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:10.836902 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6689a-55ad-4710-ace6-db111403529e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:10.837467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6689a-55ad-4710-ace6-db111403529e service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:10.837753 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6689a-55ad-4710-ace6-db111403529e service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:10.838028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6689a-55ad-4710-ace6-db111403529e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:10.847506 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6689a-55ad-4710-ace6-db111403529e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:10.848985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6689a-55ad-4710-ace6-db111403529e service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:10.860276 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1137/2271] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:13.816646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319d1eb-aa47-4967-8e92-9ea162a988c8 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:13.818184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4319d1eb-aa47-4967-8e92-9ea162a988c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:13.893215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319d1eb-aa47-4967-8e92-9ea162a988c8 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:13.900906 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e17a01-658d-449a-8e70-cca3f5312cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:13.901807 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e17a01-658d-449a-8e70-cca3f5312cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:13.902095 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e17a01-658d-449a-8e70-cca3f5312cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:13.902374 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e17a01-658d-449a-8e70-cca3f5312cc2 admin admin] PATH_INFO: `/v3/users/5d4f3f34b63942f88b970f2fa06dc258` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:13.905192 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e17a01-658d-449a-8e70-cca3f5312cc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d4f3f34b63942f88b970f2fa06dc258)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:13.907402 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e17a01-658d-449a-8e70-cca3f5312cc2 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:13.925903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6689a-55ad-4710-ace6-db111403529e service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:13.939500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6689a-55ad-4710-ace6-db111403529e service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:13.947354 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d95e43-c504-42dd-a2d4-d2463aa69a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:13.948746 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d95e43-c504-42dd-a2d4-d2463aa69a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:13.949146 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d95e43-c504-42dd-a2d4-d2463aa69a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:13.949661 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d95e43-c504-42dd-a2d4-d2463aa69a16 admin admin] PATH_INFO: `/v3/users/659f14cafb0848b8852fe4db9dd7a681` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:13.951473 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d95e43-c504-42dd-a2d4-d2463aa69a16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=659f14cafb0848b8852fe4db9dd7a681)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:13.954833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d95e43-c504-42dd-a2d4-d2463aa69a16 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:13.971463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:13.989581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-76e17a01-658d-449a-8e70-cca3f5312cc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d4f3f34b63942f88b970f2fa06dc258'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.991636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:14.005339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-76e17a01-658d-449a-8e70-cca3f5312cc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d4f3f34b63942f88b970f2fa06dc258'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:14.042677 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-76e17a01-658d-449a-8e70-cca3f5312cc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d4f3f34b63942f88b970f2fa06dc258'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:14.056951 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-76e17a01-658d-449a-8e70-cca3f5312cc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d4f3f34b63942f88b970f2fa06dc258'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:14.059008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-76e17a01-658d-449a-8e70-cca3f5312cc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d4f3f34b63942f88b970f2fa06dc258'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:14.060281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b0d95e43-c504-42dd-a2d4-d2463aa69a16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '659f14cafb0848b8852fe4db9dd7a681'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:14.075084 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b0d95e43-c504-42dd-a2d4-d2463aa69a16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '659f14cafb0848b8852fe4db9dd7a681'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:14.075323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1135/2272] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:26:13 2022] DELETE /identity/v3/users/5d4f3f34b63942f88b970f2fa06dc258 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:14.078558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b0d95e43-c504-42dd-a2d4-d2463aa69a16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '659f14cafb0848b8852fe4db9dd7a681'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:14.081671 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fc53f-3b40-4e0b-9a4b-636630519f34 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:14.082071 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fc53f-3b40-4e0b-9a4b-636630519f34 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:14.082580 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fc53f-3b40-4e0b-9a4b-636630519f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:14.094206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b0d95e43-c504-42dd-a2d4-d2463aa69a16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '659f14cafb0848b8852fe4db9dd7a681'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:14.108200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-345fc53f-3b40-4e0b-9a4b-636630519f34 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:14.109596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-345fc53f-3b40-4e0b-9a4b-636630519f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:14.138682 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b0d95e43-c504-42dd-a2d4-d2463aa69a16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '659f14cafb0848b8852fe4db9dd7a681'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:14.151996 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1138/2273] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:26:13 2022] DELETE /identity/v3/users/659f14cafb0848b8852fe4db9dd7a681 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:14.156358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24472207-a06e-4601-beea-c3d8e71084a5 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:14.156661 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24472207-a06e-4601-beea-c3d8e71084a5 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:14.157073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24472207-a06e-4601-beea-c3d8e71084a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:14.185313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-24472207-a06e-4601-beea-c3d8e71084a5 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:14.186160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-24472207-a06e-4601-beea-c3d8e71084a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:14.208091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1136/2274] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:14.223464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345fc53f-3b40-4e0b-9a4b-636630519f34 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:14.227799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-345fc53f-3b40-4e0b-9a4b-636630519f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:14.280627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1139/2275] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:26:14.293812 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24472207-a06e-4601-beea-c3d8e71084a5 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:14.298211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-24472207-a06e-4601-beea-c3d8e71084a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:14.327193 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345fc53f-3b40-4e0b-9a4b-636630519f34 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:14.401812 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24472207-a06e-4601-beea-c3d8e71084a5 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:14.406366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a40a53-adb5-47e3-98f7-239e5fcd5d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:14.407043 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e7128-0518-4b5c-ad9c-9ab24024c9a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:14.407220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a40a53-adb5-47e3-98f7-239e5fcd5d10 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:14.407467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a40a53-adb5-47e3-98f7-239e5fcd5d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:14.407710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e7128-0518-4b5c-ad9c-9ab24024c9a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:14.407824 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a40a53-adb5-47e3-98f7-239e5fcd5d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:14.408009 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e7128-0518-4b5c-ad9c-9ab24024c9a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:14.408266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e7128-0518-4b5c-ad9c-9ab24024c9a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:14.418816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e7128-0518-4b5c-ad9c-9ab24024c9a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:14.419036 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a40a53-adb5-47e3-98f7-239e5fcd5d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:14.420256 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e7128-0518-4b5c-ad9c-9ab24024c9a6 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:14.420422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a40a53-adb5-47e3-98f7-239e5fcd5d10 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:14.432848 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1137/2276] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:14.434869 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1140/2277] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:15.031177 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e7128-0518-4b5c-ad9c-9ab24024c9a6 service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:15.033630 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5d9e7128-0518-4b5c-ad9c-9ab24024c9a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:15.061472 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e7128-0518-4b5c-ad9c-9ab24024c9a6 service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:15.069822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03095e-ace8-41ab-8cc0-3e5921b852a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:15.071823 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03095e-ace8-41ab-8cc0-3e5921b852a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:15.072498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03095e-ace8-41ab-8cc0-3e5921b852a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:15.072937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03095e-ace8-41ab-8cc0-3e5921b852a5 admin admin] PATH_INFO: `/v3/projects/dd21ead5b3d24e43bcd1ab7bbda9a7da` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:15.075388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a40a53-adb5-47e3-98f7-239e5fcd5d10 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:15.077206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e5a40a53-adb5-47e3-98f7-239e5fcd5d10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:15.087559 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03095e-ace8-41ab-8cc0-3e5921b852a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd21ead5b3d24e43bcd1ab7bbda9a7da)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:15.092320 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03095e-ace8-41ab-8cc0-3e5921b852a5 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:15.120062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a40a53-adb5-47e3-98f7-239e5fcd5d10 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:15.125866 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e5448-5218-4970-b144-6721ceee2807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:15.127175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e5448-5218-4970-b144-6721ceee2807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:15.127896 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e5448-5218-4970-b144-6721ceee2807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:15.128390 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e5448-5218-4970-b144-6721ceee2807 admin admin] PATH_INFO: `/v3/projects/00b09a6b498b4ce89e3e31cc841aa945` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:15.129969 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e5448-5218-4970-b144-6721ceee2807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00b09a6b498b4ce89e3e31cc841aa945)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:15.132398 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e5448-5218-4970-b144-6721ceee2807 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:15.171784 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:15.191412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:15.245660 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4d03095e-ace8-41ab-8cc0-3e5921b852a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd21ead5b3d24e43bcd1ab7bbda9a7da', 'request_id': 'req-4d03095e-ace8-41ab-8cc0-3e5921b852a5'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.255718 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-580e5448-5218-4970-b144-6721ceee2807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00b09a6b498b4ce89e3e31cc841aa945', 'request_id': 'req-580e5448-5218-4970-b144-6721ceee2807'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.265133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-580e5448-5218-4970-b144-6721ceee2807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00b09a6b498b4ce89e3e31cc841aa945', 'request_id': 'req-580e5448-5218-4970-b144-6721ceee2807'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.273853 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4d03095e-ace8-41ab-8cc0-3e5921b852a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd21ead5b3d24e43bcd1ab7bbda9a7da', 'request_id': 'req-4d03095e-ace8-41ab-8cc0-3e5921b852a5'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.282376 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4d03095e-ace8-41ab-8cc0-3e5921b852a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd21ead5b3d24e43bcd1ab7bbda9a7da', 'request_id': 'req-4d03095e-ace8-41ab-8cc0-3e5921b852a5'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.285534 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-580e5448-5218-4970-b144-6721ceee2807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00b09a6b498b4ce89e3e31cc841aa945', 'request_id': 'req-580e5448-5218-4970-b144-6721ceee2807'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.305938 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-580e5448-5218-4970-b144-6721ceee2807 admin admin] The token cache is being invalidate because project 00b09a6b498b4ce89e3e31cc841aa945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:26:15.306230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-580e5448-5218-4970-b144-6721ceee2807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.308141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-580e5448-5218-4970-b144-6721ceee2807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.315465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1141/2278] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:26:15 2022] DELETE /identity/v3/projects/00b09a6b498b4ce89e3e31cc841aa945 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:15.316625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e5448-5218-4970-b144-6721ceee2807 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:15.318090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-580e5448-5218-4970-b144-6721ceee2807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:15.324440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4d03095e-ace8-41ab-8cc0-3e5921b852a5 admin admin] The token cache is being invalidate because project dd21ead5b3d24e43bcd1ab7bbda9a7da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:26:15.325057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4d03095e-ace8-41ab-8cc0-3e5921b852a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.326077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4d03095e-ace8-41ab-8cc0-3e5921b852a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.328159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1138/2279] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:26:15 2022] DELETE /identity/v3/projects/dd21ead5b3d24e43bcd1ab7bbda9a7da => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:15.334031 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e7721-19f7-4463-b366-2b7e964f4f45 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:15.334602 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e7721-19f7-4463-b366-2b7e964f4f45 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:15.335149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e7721-19f7-4463-b366-2b7e964f4f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:15.370993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-c39e7721-19f7-4463-b366-2b7e964f4f45 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:15.372981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c39e7721-19f7-4463-b366-2b7e964f4f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:15.473063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1139/2280] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:15.481529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e6ed7-8e98-4ff3-b9f2-0679bcaf998b None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:15.481529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e6ed7-8e98-4ff3-b9f2-0679bcaf998b None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:15.481529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e6ed7-8e98-4ff3-b9f2-0679bcaf998b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:15.513823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e5448-5218-4970-b144-6721ceee2807 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:15.514080 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-158e6ed7-8e98-4ff3-b9f2-0679bcaf998b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:15.515824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-158e6ed7-8e98-4ff3-b9f2-0679bcaf998b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:15.520843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502bc96-a51b-4719-b9ab-05ac8d212c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:15.521740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502bc96-a51b-4719-b9ab-05ac8d212c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:15.523974 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502bc96-a51b-4719-b9ab-05ac8d212c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:15.524307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502bc96-a51b-4719-b9ab-05ac8d212c08 admin admin] PATH_INFO: `/v3/users/1f9a65c34b0e4ecd8a0f5d5db62808df` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:15.526195 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502bc96-a51b-4719-b9ab-05ac8d212c08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f9a65c34b0e4ecd8a0f5d5db62808df)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:15.531010 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502bc96-a51b-4719-b9ab-05ac8d212c08 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:15.533044 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1140/2281] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:15.534222 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e6ed7-8e98-4ff3-b9f2-0679bcaf998b None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:15.547101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e6ed7-8e98-4ff3-b9f2-0679bcaf998b None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:15.555625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3542cdc-42b7-4b95-b841-b2ef02a33ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:15.557070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3542cdc-42b7-4b95-b841-b2ef02a33ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:15.557583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3542cdc-42b7-4b95-b841-b2ef02a33ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:15.558396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3542cdc-42b7-4b95-b841-b2ef02a33ac5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:15.559990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3542cdc-42b7-4b95-b841-b2ef02a33ac5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:26:15.560771 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3542cdc-42b7-4b95-b841-b2ef02a33ac5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:15.565980 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3542cdc-42b7-4b95-b841-b2ef02a33ac5 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:15.586368 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1141/2282] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:26:15.591288 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3542cdc-42b7-4b95-b841-b2ef02a33ac5 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:15.611595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3542cdc-42b7-4b95-b841-b2ef02a33ac5 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:15.613936 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41678913-644d-42cb-b327-5e3c51074197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:15.616249 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41678913-644d-42cb-b327-5e3c51074197 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:15.616666 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41678913-644d-42cb-b327-5e3c51074197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:15.617065 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41678913-644d-42cb-b327-5e3c51074197 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:15.617984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41678913-644d-42cb-b327-5e3c51074197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:15.620164 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41678913-644d-42cb-b327-5e3c51074197 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:15.637508 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-c502bc96-a51b-4719-b9ab-05ac8d212c08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f9a65c34b0e4ecd8a0f5d5db62808df'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.647377 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-c502bc96-a51b-4719-b9ab-05ac8d212c08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f9a65c34b0e4ecd8a0f5d5db62808df'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.649763 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-c502bc96-a51b-4719-b9ab-05ac8d212c08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f9a65c34b0e4ecd8a0f5d5db62808df'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.659121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1142/2283] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:26:15 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:26:15.660317 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41678913-644d-42cb-b327-5e3c51074197 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:15.662412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-c502bc96-a51b-4719-b9ab-05ac8d212c08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f9a65c34b0e4ecd8a0f5d5db62808df'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.664515 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-41678913-644d-42cb-b327-5e3c51074197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:15.688064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-c502bc96-a51b-4719-b9ab-05ac8d212c08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f9a65c34b0e4ecd8a0f5d5db62808df'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.709863 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1142/2284] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:26:15 2022] DELETE /identity/v3/users/1f9a65c34b0e4ecd8a0f5d5db62808df => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:15.775720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41678913-644d-42cb-b327-5e3c51074197 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:15.788006 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:15.873359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ac840-eb98-4408-b027-f2e967c5e4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:15.874724 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ac840-eb98-4408-b027-f2e967c5e4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:15.876366 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ac840-eb98-4408-b027-f2e967c5e4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:15.876830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ac840-eb98-4408-b027-f2e967c5e4f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:15.878223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ac840-eb98-4408-b027-f2e967c5e4f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:15.881907 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ac840-eb98-4408-b027-f2e967c5e4f1 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:15.931419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1143/2285] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:26:15 2022] POST /identity/v3/users => generated 283 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:26:15.936279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502bc96-a51b-4719-b9ab-05ac8d212c08 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:15.952664 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502bc96-a51b-4719-b9ab-05ac8d212c08 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:15.958379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555d267-6390-420d-9d5c-7e196af723e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:15.959859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555d267-6390-420d-9d5c-7e196af723e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:15.960606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555d267-6390-420d-9d5c-7e196af723e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:15.961004 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555d267-6390-420d-9d5c-7e196af723e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:15.963252 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555d267-6390-420d-9d5c-7e196af723e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:15.965214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555d267-6390-420d-9d5c-7e196af723e9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:15.985982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1143/2286] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:26:15.991822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ac840-eb98-4408-b027-f2e967c5e4f1 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:16.009554 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ac840-eb98-4408-b027-f2e967c5e4f1 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:16.016987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4561a2-2de7-4edb-9c6c-6f3bc593ecb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:16.018254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4561a2-2de7-4edb-9c6c-6f3bc593ecb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.018764 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4561a2-2de7-4edb-9c6c-6f3bc593ecb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.019272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4561a2-2de7-4edb-9c6c-6f3bc593ecb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.020843 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4561a2-2de7-4edb-9c6c-6f3bc593ecb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:16.022992 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4561a2-2de7-4edb-9c6c-6f3bc593ecb5 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:16.038291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1144/2287] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:26:16.043460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555d267-6390-420d-9d5c-7e196af723e9 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:16.059305 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555d267-6390-420d-9d5c-7e196af723e9 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:16.066010 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c79ac-3a46-4ea8-a6b6-d73854c918b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:16.067323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c79ac-3a46-4ea8-a6b6-d73854c918b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.067448 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4561a2-2de7-4edb-9c6c-6f3bc593ecb5 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:16.070271 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-bb4561a2-2de7-4edb-9c6c-6f3bc593ecb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:16.077967 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c79ac-3a46-4ea8-a6b6-d73854c918b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.078673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c79ac-3a46-4ea8-a6b6-d73854c918b0 admin admin] PATH_INFO: `/v3/projects/1d0e861e3eb6454eada1c1de4f5829bd/users/341e65ec547b4ae5a911cc7d11127f16/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.095329 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4561a2-2de7-4edb-9c6c-6f3bc593ecb5 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:16.103606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9e4b1-7711-4b3f-9bf1-bb4b55ac8339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:16.104562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9e4b1-7711-4b3f-9bf1-bb4b55ac8339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.104845 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9e4b1-7711-4b3f-9bf1-bb4b55ac8339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.105122 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9e4b1-7711-4b3f-9bf1-bb4b55ac8339 admin admin] PATH_INFO: `/v3/projects/c92db13a24b149a78d254a042b4a3b36` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.106401 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9e4b1-7711-4b3f-9bf1-bb4b55ac8339 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c92db13a24b149a78d254a042b4a3b36)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:16.108451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9e4b1-7711-4b3f-9bf1-bb4b55ac8339 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:16.117282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c79ac-3a46-4ea8-a6b6-d73854c918b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d0e861e3eb6454eada1c1de4f5829bd, user_id=341e65ec547b4ae5a911cc7d11127f16, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:16.122856 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c79ac-3a46-4ea8-a6b6-d73854c918b0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:16.128209 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:16.164256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1144/2288] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:26:16 2022] PUT /identity/v3/projects/1d0e861e3eb6454eada1c1de4f5829bd/users/341e65ec547b4ae5a911cc7d11127f16/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:26:16.167008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5badb-13b0-4004-8c93-7d4869e31c04 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.167008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5badb-13b0-4004-8c93-7d4869e31c04 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.167661 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5badb-13b0-4004-8c93-7d4869e31c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.184866 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-97f9e4b1-7711-4b3f-9bf1-bb4b55ac8339 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c92db13a24b149a78d254a042b4a3b36', 'request_id': 'req-97f9e4b1-7711-4b3f-9bf1-bb4b55ac8339'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:16.201007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-97f9e4b1-7711-4b3f-9bf1-bb4b55ac8339 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c92db13a24b149a78d254a042b4a3b36', 'request_id': 'req-97f9e4b1-7711-4b3f-9bf1-bb4b55ac8339'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:16.211389 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-97f9e4b1-7711-4b3f-9bf1-bb4b55ac8339 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c92db13a24b149a78d254a042b4a3b36', 'request_id': 'req-97f9e4b1-7711-4b3f-9bf1-bb4b55ac8339'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:16.220193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f4e5badb-13b0-4004-8c93-7d4869e31c04 None None] MFA Rules not processed for user `341e65ec547b4ae5a911cc7d11127f16`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:16.222213 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f4e5badb-13b0-4004-8c93-7d4869e31c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:16.234439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-97f9e4b1-7711-4b3f-9bf1-bb4b55ac8339 admin admin] The token cache is being invalidate because project c92db13a24b149a78d254a042b4a3b36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:26:16.234779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-97f9e4b1-7711-4b3f-9bf1-bb4b55ac8339 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:16.236014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-97f9e4b1-7711-4b3f-9bf1-bb4b55ac8339 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:16.238804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1145/2289] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:26:16 2022] DELETE /identity/v3/projects/c92db13a24b149a78d254a042b4a3b36 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:16.249391 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e339ac0-ef45-43dc-8988-d1df89f55334 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.250290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e339ac0-ef45-43dc-8988-d1df89f55334 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.251000 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e339ac0-ef45-43dc-8988-d1df89f55334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.333823 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-6e339ac0-ef45-43dc-8988-d1df89f55334 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:16.335189 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6e339ac0-ef45-43dc-8988-d1df89f55334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:16.375057 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1145/2290] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:16.386495 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5badb-13b0-4004-8c93-7d4869e31c04 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:16.389419 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f4e5badb-13b0-4004-8c93-7d4869e31c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:16.452937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1146/2291] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:16.462178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d7cad-8153-4938-ac4a-efa10b88d555 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.462664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d7cad-8153-4938-ac4a-efa10b88d555 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.463053 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d7cad-8153-4938-ac4a-efa10b88d555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.498508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4c1d7cad-8153-4938-ac4a-efa10b88d555 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:16.500200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4c1d7cad-8153-4938-ac4a-efa10b88d555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:16.509295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1147/2292] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:16.516086 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5badb-13b0-4004-8c93-7d4869e31c04 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:16.517968 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d7cad-8153-4938-ac4a-efa10b88d555 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:16.529844 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d7cad-8153-4938-ac4a-efa10b88d555 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:16.536205 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8395f-43dc-4004-8213-661d65508d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:16.537815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8395f-43dc-4004-8213-661d65508d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.538265 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8395f-43dc-4004-8213-661d65508d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.538382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d148cbe-fe28-4293-b175-d68d0a6d974c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:16.538810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8395f-43dc-4004-8213-661d65508d9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.539692 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8395f-43dc-4004-8213-661d65508d9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:26:16.540156 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d148cbe-fe28-4293-b175-d68d0a6d974c service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.540304 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8395f-43dc-4004-8213-661d65508d9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:16.540649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d148cbe-fe28-4293-b175-d68d0a6d974c service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.542274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8395f-43dc-4004-8213-661d65508d9d admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:16.542678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d148cbe-fe28-4293-b175-d68d0a6d974c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.547943 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:16.570412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d148cbe-fe28-4293-b175-d68d0a6d974c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:16.572155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d148cbe-fe28-4293-b175-d68d0a6d974c service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:16.575391 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1148/2293] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:26:16.582084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8395f-43dc-4004-8213-661d65508d9d admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:16.597142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1146/2294] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:26:16.602797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8395f-43dc-4004-8213-661d65508d9d admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:16.611194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987bf87-9e43-4c8c-a06a-1fdd3a803663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:16.612278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987bf87-9e43-4c8c-a06a-1fdd3a803663 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.612842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987bf87-9e43-4c8c-a06a-1fdd3a803663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.613599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987bf87-9e43-4c8c-a06a-1fdd3a803663 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.615291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987bf87-9e43-4c8c-a06a-1fdd3a803663 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:16.617845 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987bf87-9e43-4c8c-a06a-1fdd3a803663 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:16.666428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1149/2295] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:26:16 2022] POST /identity/v3/projects => generated 369 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:26:16.667499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d148cbe-fe28-4293-b175-d68d0a6d974c service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:16.685585 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d148cbe-fe28-4293-b175-d68d0a6d974c service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:16.828169 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d90db-52bd-4094-bb4d-6da3d1d29d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:16.829608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d90db-52bd-4094-bb4d-6da3d1d29d44 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.830479 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d90db-52bd-4094-bb4d-6da3d1d29d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.831119 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d90db-52bd-4094-bb4d-6da3d1d29d44 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.838173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d90db-52bd-4094-bb4d-6da3d1d29d44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:16.840599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d90db-52bd-4094-bb4d-6da3d1d29d44 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:16.867690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758c193-8623-4327-b6b7-eb2872d22e56 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.868245 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758c193-8623-4327-b6b7-eb2872d22e56 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.868787 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758c193-8623-4327-b6b7-eb2872d22e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.884377 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1147/2296] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:26:16 2022] POST /identity/v3/users => generated 285 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:26:16.890721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d90db-52bd-4094-bb4d-6da3d1d29d44 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:16.898474 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-6758c193-8623-4327-b6b7-eb2872d22e56 None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:16.901306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6758c193-8623-4327-b6b7-eb2872d22e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:16.922458 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d90db-52bd-4094-bb4d-6da3d1d29d44 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:16.932841 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ec758-a6b8-4f04-b596-5285964b257a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:16.933618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ec758-a6b8-4f04-b596-5285964b257a admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.933903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ec758-a6b8-4f04-b596-5285964b257a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.934181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ec758-a6b8-4f04-b596-5285964b257a admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.935580 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ec758-a6b8-4f04-b596-5285964b257a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:16.950575 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ec758-a6b8-4f04-b596-5285964b257a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:17.028492 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1148/2297] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:26:17.034410 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ec758-a6b8-4f04-b596-5285964b257a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:17.060985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ec758-a6b8-4f04-b596-5285964b257a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:17.077349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7308db7-d877-40b6-8a41-3428a2758829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:17.078150 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7308db7-d877-40b6-8a41-3428a2758829 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:17.078756 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7308db7-d877-40b6-8a41-3428a2758829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:17.078756 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7308db7-d877-40b6-8a41-3428a2758829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:17.080441 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7308db7-d877-40b6-8a41-3428a2758829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:17.084606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7308db7-d877-40b6-8a41-3428a2758829 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:17.087958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:17.124464 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1149/2298] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:26:17.141530 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7308db7-d877-40b6-8a41-3428a2758829 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:17.187887 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7308db7-d877-40b6-8a41-3428a2758829 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:17.226796 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7720679-66d4-48e1-b1eb-c6a80f0a7be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:17.229200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7720679-66d4-48e1-b1eb-c6a80f0a7be5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:17.229673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7720679-66d4-48e1-b1eb-c6a80f0a7be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:17.230157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7720679-66d4-48e1-b1eb-c6a80f0a7be5 admin admin] PATH_INFO: `/v3/projects/8e836e6c6faf4473941b42d5a978b88e/users/764c95c9af6c4b67b743b569b1696743/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:17.254096 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7720679-66d4-48e1-b1eb-c6a80f0a7be5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e836e6c6faf4473941b42d5a978b88e, user_id=764c95c9af6c4b67b743b569b1696743, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:17.258457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1150/2299] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jul 20 19:26:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:26:17.259392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7720679-66d4-48e1-b1eb-c6a80f0a7be5 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:17.278012 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758c193-8623-4327-b6b7-eb2872d22e56 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:17.281476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6758c193-8623-4327-b6b7-eb2872d22e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:17.291695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1150/2300] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:26:17 2022] PUT /identity/v3/projects/8e836e6c6faf4473941b42d5a978b88e/users/764c95c9af6c4b67b743b569b1696743/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:17.381942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ddb92-7435-4d87-87d9-2f7fe1d9e3ae None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:17.382344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ddb92-7435-4d87-87d9-2f7fe1d9e3ae None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:17.382636 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ddb92-7435-4d87-87d9-2f7fe1d9e3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:17.465865 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-640ddb92-7435-4d87-87d9-2f7fe1d9e3ae None None] MFA Rules not processed for user `764c95c9af6c4b67b743b569b1696743`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:17.471272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-640ddb92-7435-4d87-87d9-2f7fe1d9e3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:17.486477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758c193-8623-4327-b6b7-eb2872d22e56 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:17.581647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fec766-6892-4ccd-834d-db3dcd925d04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:17.582430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fec766-6892-4ccd-834d-db3dcd925d04 service swift] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:17.582773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fec766-6892-4ccd-834d-db3dcd925d04 service swift] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:17.583163 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fec766-6892-4ccd-834d-db3dcd925d04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:17.623572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1151/2301] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:17.640603 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:17.725494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fec766-6892-4ccd-834d-db3dcd925d04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:17.727121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fec766-6892-4ccd-834d-db3dcd925d04 service swift] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:17.738153 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1151/2302] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:26:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 19:26:19.786209 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d7144-b6cc-4b8a-9ac2-15773c32723f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:19.788409 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d7144-b6cc-4b8a-9ac2-15773c32723f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:19.788868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d7144-b6cc-4b8a-9ac2-15773c32723f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:19.837617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f76d7144-b6cc-4b8a-9ac2-15773c32723f None None] MFA Rules not processed for user `cb06eeba738b452fab3986de7e1f7856`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:19.838935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f76d7144-b6cc-4b8a-9ac2-15773c32723f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:20.079588 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1152/2303] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:27.734010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=17 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:29.076960 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fec766-6892-4ccd-834d-db3dcd925d04 service swift] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:29.078465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c6fec766-6892-4ccd-834d-db3dcd925d04 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:29.169070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fec766-6892-4ccd-834d-db3dcd925d04 service swift] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:29.173514 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd581e-7a61-43c6-b926-0490143d2dc0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3e28557896e41c893f0de2590bc96d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:29.174392 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd581e-7a61-43c6-b926-0490143d2dc0 service glance] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:29.174659 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd581e-7a61-43c6-b926-0490143d2dc0 service glance] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:29.175003 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd581e-7a61-43c6-b926-0490143d2dc0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:29.185221 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd581e-7a61-43c6-b926-0490143d2dc0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:29.187060 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd581e-7a61-43c6-b926-0490143d2dc0 service glance] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:29.197608 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1152/2304] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:29.964788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=69,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:30.278333 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e9c03-0d3d-4050-893b-e377777c43f9 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:30.278777 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e9c03-0d3d-4050-893b-e377777c43f9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:30.279124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e9c03-0d3d-4050-893b-e377777c43f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:30.299453 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c19e9c03-0d3d-4050-893b-e377777c43f9 None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:30.300936 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c19e9c03-0d3d-4050-893b-e377777c43f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:30.310325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1153/2305] 10.222.0.43 () {58 vars in 967 bytes} [Wed Jul 20 19:26:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:30.328124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd581e-7a61-43c6-b926-0490143d2dc0 service glance] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:30.349488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd581e-7a61-43c6-b926-0490143d2dc0 service glance] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:30.358331 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69ea79-bc98-4744-bc5d-97784056def5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:30.359281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69ea79-bc98-4744-bc5d-97784056def5 service swift] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:30.359779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69ea79-bc98-4744-bc5d-97784056def5 service swift] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:30.363477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69ea79-bc98-4744-bc5d-97784056def5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:30.374832 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a69ea79-bc98-4744-bc5d-97784056def5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:30.377406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a69ea79-bc98-4744-bc5d-97784056def5 service swift] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:30.387861 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1153/2306] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:26:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:26:32.579898 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e9c03-0d3d-4050-893b-e377777c43f9 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:32.581416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c19e9c03-0d3d-4050-893b-e377777c43f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:32.649403 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e9c03-0d3d-4050-893b-e377777c43f9 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:32.654202 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f2bb5-bb7b-4e9e-9517-cb118695003d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:32.655232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f2bb5-bb7b-4e9e-9517-cb118695003d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:32.655656 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f2bb5-bb7b-4e9e-9517-cb118695003d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:32.655943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f2bb5-bb7b-4e9e-9517-cb118695003d admin admin] PATH_INFO: `/v3/users/83ba05eb66ec44598ec0809cfc0c4ccd` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:32.657367 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f2bb5-bb7b-4e9e-9517-cb118695003d admin admin] RBAC: Authorizing `identity:delete_user(user_id=83ba05eb66ec44598ec0809cfc0c4ccd)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:32.659541 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f2bb5-bb7b-4e9e-9517-cb118695003d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:32.707809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-061f2bb5-bb7b-4e9e-9517-cb118695003d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83ba05eb66ec44598ec0809cfc0c4ccd'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:32.716238 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-061f2bb5-bb7b-4e9e-9517-cb118695003d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83ba05eb66ec44598ec0809cfc0c4ccd'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:32.716947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-061f2bb5-bb7b-4e9e-9517-cb118695003d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83ba05eb66ec44598ec0809cfc0c4ccd'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:32.725482 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-061f2bb5-bb7b-4e9e-9517-cb118695003d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83ba05eb66ec44598ec0809cfc0c4ccd'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:32.738302 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-061f2bb5-bb7b-4e9e-9517-cb118695003d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83ba05eb66ec44598ec0809cfc0c4ccd'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:32.744876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1154/2307] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:26:32 2022] DELETE /identity/v3/users/83ba05eb66ec44598ec0809cfc0c4ccd => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:32.924667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69ea79-bc98-4744-bc5d-97784056def5 service swift] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:32.938699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69ea79-bc98-4744-bc5d-97784056def5 service swift] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:32.998470 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de88ea9-8eb7-435f-b564-441c57d5de32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:32.999338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de88ea9-8eb7-435f-b564-441c57d5de32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:32.999626 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de88ea9-8eb7-435f-b564-441c57d5de32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:32.999901 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de88ea9-8eb7-435f-b564-441c57d5de32 admin admin] PATH_INFO: `/v3/projects/aba8221c6e144e94ae026555d5317576` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:33.001124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de88ea9-8eb7-435f-b564-441c57d5de32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba8221c6e144e94ae026555d5317576)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:33.003033 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de88ea9-8eb7-435f-b564-441c57d5de32 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:33.053517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-8de88ea9-8eb7-435f-b564-441c57d5de32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba8221c6e144e94ae026555d5317576', 'request_id': 'req-8de88ea9-8eb7-435f-b564-441c57d5de32'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:33.065158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-8de88ea9-8eb7-435f-b564-441c57d5de32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba8221c6e144e94ae026555d5317576', 'request_id': 'req-8de88ea9-8eb7-435f-b564-441c57d5de32'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:33.087096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-8de88ea9-8eb7-435f-b564-441c57d5de32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba8221c6e144e94ae026555d5317576', 'request_id': 'req-8de88ea9-8eb7-435f-b564-441c57d5de32'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:33.112999 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-8de88ea9-8eb7-435f-b564-441c57d5de32 admin admin] The token cache is being invalidate because project aba8221c6e144e94ae026555d5317576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:26:33.113303 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-8de88ea9-8eb7-435f-b564-441c57d5de32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:33.115384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-8de88ea9-8eb7-435f-b564-441c57d5de32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:33.120327 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1154/2308] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:26:32 2022] DELETE /identity/v3/projects/aba8221c6e144e94ae026555d5317576 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:33.122799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f5c70-2775-45e2-9cde-cb9d79d2d2bc None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:33.123320 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f5c70-2775-45e2-9cde-cb9d79d2d2bc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:33.123601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f5c70-2775-45e2-9cde-cb9d79d2d2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:33.147920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-853f5c70-2775-45e2-9cde-cb9d79d2d2bc None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:33.149187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-853f5c70-2775-45e2-9cde-cb9d79d2d2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:33.303412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1155/2309] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:33.305880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d4aa4-6654-43e7-89a0-3180158c231f None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:33.306179 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d4aa4-6654-43e7-89a0-3180158c231f None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:33.306463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d4aa4-6654-43e7-89a0-3180158c231f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:33.344657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-839d4aa4-6654-43e7-89a0-3180158c231f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:33.355378 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-839d4aa4-6654-43e7-89a0-3180158c231f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:33.382293 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1155/2310] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:33.386094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f5c70-2775-45e2-9cde-cb9d79d2d2bc None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:33.396150 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f5c70-2775-45e2-9cde-cb9d79d2d2bc None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:33.401604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258371c5-7326-43d0-a9cd-ccd073b0a106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:33.402364 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258371c5-7326-43d0-a9cd-ccd073b0a106 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:33.402845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258371c5-7326-43d0-a9cd-ccd073b0a106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:33.403189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258371c5-7326-43d0-a9cd-ccd073b0a106 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:33.404024 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258371c5-7326-43d0-a9cd-ccd073b0a106 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:26:33.404420 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258371c5-7326-43d0-a9cd-ccd073b0a106 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:33.406084 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258371c5-7326-43d0-a9cd-ccd073b0a106 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:33.415555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:33.419388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1156/2311] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:26:33.423967 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d4aa4-6654-43e7-89a0-3180158c231f None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:33.435006 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d4aa4-6654-43e7-89a0-3180158c231f None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:33.440008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282911cb-e735-4b1b-a80d-5cb881116d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:33.440721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282911cb-e735-4b1b-a80d-5cb881116d23 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:33.441006 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282911cb-e735-4b1b-a80d-5cb881116d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:33.441279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282911cb-e735-4b1b-a80d-5cb881116d23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:33.441982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282911cb-e735-4b1b-a80d-5cb881116d23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:33.444642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282911cb-e735-4b1b-a80d-5cb881116d23 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:33.454768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:33.474087 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1156/2312] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:26:33 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:26:33.478598 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258371c5-7326-43d0-a9cd-ccd073b0a106 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:33.495839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258371c5-7326-43d0-a9cd-ccd073b0a106 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:33.587506 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba409d1-5dea-4337-87e5-3daa20505a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:33.588497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba409d1-5dea-4337-87e5-3daa20505a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:33.588809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba409d1-5dea-4337-87e5-3daa20505a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:33.589155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba409d1-5dea-4337-87e5-3daa20505a2a admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:33.589942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba409d1-5dea-4337-87e5-3daa20505a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:33.591952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba409d1-5dea-4337-87e5-3daa20505a2a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:33.619402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1157/2313] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:26:33 2022] POST /identity/v3/users => generated 287 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:26:33.623505 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282911cb-e735-4b1b-a80d-5cb881116d23 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:33.635959 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282911cb-e735-4b1b-a80d-5cb881116d23 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:33.641853 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c67493-c13d-42af-ade0-1a5b031efd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:33.642528 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c67493-c13d-42af-ade0-1a5b031efd35 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:33.642972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c67493-c13d-42af-ade0-1a5b031efd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:33.643257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c67493-c13d-42af-ade0-1a5b031efd35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:33.644350 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c67493-c13d-42af-ade0-1a5b031efd35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:33.646003 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c67493-c13d-42af-ade0-1a5b031efd35 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:33.658342 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1157/2314] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:26:33.662751 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba409d1-5dea-4337-87e5-3daa20505a2a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:33.674802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba409d1-5dea-4337-87e5-3daa20505a2a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:33.680744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77b2a2-7712-47ff-865a-33c2b5439f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:33.681859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77b2a2-7712-47ff-865a-33c2b5439f8a admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:33.681859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77b2a2-7712-47ff-865a-33c2b5439f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:33.682222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77b2a2-7712-47ff-865a-33c2b5439f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:33.682723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77b2a2-7712-47ff-865a-33c2b5439f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:33.684750 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77b2a2-7712-47ff-865a-33c2b5439f8a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:33.697898 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1158/2315] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:26:33.702158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c67493-c13d-42af-ade0-1a5b031efd35 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:33.715573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c67493-c13d-42af-ade0-1a5b031efd35 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:33.720971 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016e30-4eb1-49a5-b31a-5d202c81c920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:33.722076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016e30-4eb1-49a5-b31a-5d202c81c920 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:33.722210 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016e30-4eb1-49a5-b31a-5d202c81c920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:33.722434 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016e30-4eb1-49a5-b31a-5d202c81c920 admin admin] PATH_INFO: `/v3/projects/2765674134b8415f8ab79fc7295fbe1d/users/a1ceda5bed824184b10bfaff48ee006c/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:33.739121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2016e30-4eb1-49a5-b31a-5d202c81c920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2765674134b8415f8ab79fc7295fbe1d, user_id=a1ceda5bed824184b10bfaff48ee006c, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:33.742405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2016e30-4eb1-49a5-b31a-5d202c81c920 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:33.757547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1158/2316] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:26:33 2022] PUT /identity/v3/projects/2765674134b8415f8ab79fc7295fbe1d/users/a1ceda5bed824184b10bfaff48ee006c/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:33.774133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd47faf-83ec-4d8c-bea9-2d1ed126c566 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:33.774426 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd47faf-83ec-4d8c-bea9-2d1ed126c566 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:33.774706 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd47faf-83ec-4d8c-bea9-2d1ed126c566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:33.806641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0dd47faf-83ec-4d8c-bea9-2d1ed126c566 None None] MFA Rules not processed for user `a1ceda5bed824184b10bfaff48ee006c`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:33.807858 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0dd47faf-83ec-4d8c-bea9-2d1ed126c566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:33.922367 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1159/2317] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:33.964049 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016e30-4eb1-49a5-b31a-5d202c81c920 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:33.967050 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e2016e30-4eb1-49a5-b31a-5d202c81c920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:34.070067 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016e30-4eb1-49a5-b31a-5d202c81c920 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:34.078918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc6bdc-ffe2-4ec1-9e8f-e252742c7e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:34.080819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc6bdc-ffe2-4ec1-9e8f-e252742c7e22 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:34.081636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc6bdc-ffe2-4ec1-9e8f-e252742c7e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:34.082122 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc6bdc-ffe2-4ec1-9e8f-e252742c7e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:34.097266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc6bdc-ffe2-4ec1-9e8f-e252742c7e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:34.105343 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc6bdc-ffe2-4ec1-9e8f-e252742c7e22 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:34.123180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1159/2318] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:34.859030 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a270f86-0bf0-4418-b1bc-f920b162c729 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:34.859568 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a270f86-0bf0-4418-b1bc-f920b162c729 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:34.860152 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a270f86-0bf0-4418-b1bc-f920b162c729 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:34.861691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1160/2319] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jul 20 19:26:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:26:34.872927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0fb2c-aa51-4f45-91ee-3ba12e189d2e None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:34.873438 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0fb2c-aa51-4f45-91ee-3ba12e189d2e None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:34.873834 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0fb2c-aa51-4f45-91ee-3ba12e189d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:34.898230 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-49b0fb2c-aa51-4f45-91ee-3ba12e189d2e None None] MFA Rules not processed for user `267b178236e046e4bc694b2da6f86eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:34.906315 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-49b0fb2c-aa51-4f45-91ee-3ba12e189d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:35.048475 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1160/2320] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jul 20 19:26:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:35.067312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a270f86-0bf0-4418-b1bc-f920b162c729 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:35.076115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0a270f86-0bf0-4418-b1bc-f920b162c729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:35.217231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a270f86-0bf0-4418-b1bc-f920b162c729 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:35.225556 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d34323-9784-451f-86af-988fa3b6134a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3e28557896e41c893f0de2590bc96d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:35.226552 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d34323-9784-451f-86af-988fa3b6134a service glance] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:35.227417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d34323-9784-451f-86af-988fa3b6134a service glance] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:35.228104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d34323-9784-451f-86af-988fa3b6134a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:35.258419 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d34323-9784-451f-86af-988fa3b6134a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:35.260172 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d34323-9784-451f-86af-988fa3b6134a service glance] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:35.300654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1161/2321] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:35.402079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777be699-a494-42cc-abc0-658f24904824 None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:35.402621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777be699-a494-42cc-abc0-658f24904824 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:35.403079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777be699-a494-42cc-abc0-658f24904824 None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:35.404890 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1161/2322] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jul 20 19:26:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:26:35.416180 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5877f-e7e8-4c3f-ae88-7fb0a487c61e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:35.416656 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5877f-e7e8-4c3f-ae88-7fb0a487c61e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:35.417032 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5877f-e7e8-4c3f-ae88-7fb0a487c61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:35.452950 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-98b5877f-e7e8-4c3f-ae88-7fb0a487c61e None None] MFA Rules not processed for user `74327a10eaab43e0af02050da1a24f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:35.454414 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-98b5877f-e7e8-4c3f-ae88-7fb0a487c61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:35.465289 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:35.600793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1162/2323] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jul 20 19:26:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:35.635394 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777be699-a494-42cc-abc0-658f24904824 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:35.637788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-777be699-a494-42cc-abc0-658f24904824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:35.653824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777be699-a494-42cc-abc0-658f24904824 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:35.661585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04587d8e-788f-480b-bf24-d02080636a26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:35.662328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04587d8e-788f-480b-bf24-d02080636a26 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:35.662706 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04587d8e-788f-480b-bf24-d02080636a26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:35.663058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04587d8e-788f-480b-bf24-d02080636a26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:35.671422 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04587d8e-788f-480b-bf24-d02080636a26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:35.673065 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04587d8e-788f-480b-bf24-d02080636a26 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:35.696634 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987845d-7537-40db-b9d6-f40b91112f9b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:35.697152 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987845d-7537-40db-b9d6-f40b91112f9b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:35.698241 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987845d-7537-40db-b9d6-f40b91112f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:35.711407 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1162/2324] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:35.743573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f987845d-7537-40db-b9d6-f40b91112f9b None None] MFA Rules not processed for user `a1ceda5bed824184b10bfaff48ee006c`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:35.745209 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f987845d-7537-40db-b9d6-f40b91112f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:35.767661 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1163/2325] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:26:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:35.842839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fffbf-78fb-4f55-b884-6aa22708fb1b None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:35.843371 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fffbf-78fb-4f55-b884-6aa22708fb1b None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:35.843798 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fffbf-78fb-4f55-b884-6aa22708fb1b None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:35.846555 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1163/2326] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jul 20 19:26:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:26:35.861587 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4c363-9fdb-4215-a7e5-4ce08f407644 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:35.862155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4c363-9fdb-4215-a7e5-4ce08f407644 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:35.862608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4c363-9fdb-4215-a7e5-4ce08f407644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:35.891788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f0c4c363-9fdb-4215-a7e5-4ce08f407644 None None] MFA Rules not processed for user `267b178236e046e4bc694b2da6f86eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:35.893117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f0c4c363-9fdb-4215-a7e5-4ce08f407644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:35.906986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1164/2327] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jul 20 19:26:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:35.922990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fffbf-78fb-4f55-b884-6aa22708fb1b None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:35.947365 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fffbf-78fb-4f55-b884-6aa22708fb1b None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:35.958348 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99062e21-0249-4ad4-8d07-6dc48e9670e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:35.959321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99062e21-0249-4ad4-8d07-6dc48e9670e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:35.959757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99062e21-0249-4ad4-8d07-6dc48e9670e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:35.960182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99062e21-0249-4ad4-8d07-6dc48e9670e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:35.979482 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99062e21-0249-4ad4-8d07-6dc48e9670e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:35.988715 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99062e21-0249-4ad4-8d07-6dc48e9670e5 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:36.009159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1164/2328] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:45.707153 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757e01b-bfe2-4daf-9efc-8aff23d230c6 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:45.707690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757e01b-bfe2-4daf-9efc-8aff23d230c6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:45.708028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757e01b-bfe2-4daf-9efc-8aff23d230c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:45.726716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c757e01b-bfe2-4daf-9efc-8aff23d230c6 None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:45.727924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c757e01b-bfe2-4daf-9efc-8aff23d230c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:45.791440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1165/2329] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jul 20 19:26:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:45.804722 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99062e21-0249-4ad4-8d07-6dc48e9670e5 service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:45.806183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-99062e21-0249-4ad4-8d07-6dc48e9670e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:45.843197 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:45.909366 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99062e21-0249-4ad4-8d07-6dc48e9670e5 service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:45.914158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f0d91-aa95-40cb-9286-08cf8829b958 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:45.914731 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f0d91-aa95-40cb-9286-08cf8829b958 service swift] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:45.915258 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f0d91-aa95-40cb-9286-08cf8829b958 service swift] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:45.915258 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f0d91-aa95-40cb-9286-08cf8829b958 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:45.924368 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f0d91-aa95-40cb-9286-08cf8829b958 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:45.925945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f0d91-aa95-40cb-9286-08cf8829b958 service swift] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:45.935133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1165/2330] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:26:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:26:55.783801 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=49 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:55.931697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=21 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:57.048713 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757e01b-bfe2-4daf-9efc-8aff23d230c6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:57.050826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c757e01b-bfe2-4daf-9efc-8aff23d230c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:57.148002 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757e01b-bfe2-4daf-9efc-8aff23d230c6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:57.153121 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60cfe0-b7fa-4673-bb60-4cfd864406b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae581d3dda354e07894cb5205d1b2c5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:57.153794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60cfe0-b7fa-4673-bb60-4cfd864406b4 service placement] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:57.154058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60cfe0-b7fa-4673-bb60-4cfd864406b4 service placement] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:57.154332 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60cfe0-b7fa-4673-bb60-4cfd864406b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:57.156120 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2f60cfe0-b7fa-4673-bb60-4cfd864406b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:57.168730 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60cfe0-b7fa-4673-bb60-4cfd864406b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:57.170422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60cfe0-b7fa-4673-bb60-4cfd864406b4 service placement] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:57.181926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1166/2331] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:07.176650 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=32 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:11.246240 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f0d91-aa95-40cb-9286-08cf8829b958 service swift] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:11.248374 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-a92f0d91-aa95-40cb-9286-08cf8829b958 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:11.345167 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f0d91-aa95-40cb-9286-08cf8829b958 service swift] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:11.355294 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe65c20-fc77-4fd6-8aef-8ecb44cf7140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:11.358284 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe65c20-fc77-4fd6-8aef-8ecb44cf7140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:11.358937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe65c20-fc77-4fd6-8aef-8ecb44cf7140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:11.359507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe65c20-fc77-4fd6-8aef-8ecb44cf7140 admin admin] PATH_INFO: `/v3/users/341e65ec547b4ae5a911cc7d11127f16` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:11.361765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe65c20-fc77-4fd6-8aef-8ecb44cf7140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=341e65ec547b4ae5a911cc7d11127f16)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:11.366350 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe65c20-fc77-4fd6-8aef-8ecb44cf7140 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:11.423400 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-7fe65c20-fc77-4fd6-8aef-8ecb44cf7140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '341e65ec547b4ae5a911cc7d11127f16'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:11.441398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-7fe65c20-fc77-4fd6-8aef-8ecb44cf7140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '341e65ec547b4ae5a911cc7d11127f16'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:11.459008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-7fe65c20-fc77-4fd6-8aef-8ecb44cf7140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '341e65ec547b4ae5a911cc7d11127f16'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:11.468240 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-7fe65c20-fc77-4fd6-8aef-8ecb44cf7140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '341e65ec547b4ae5a911cc7d11127f16'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:11.469227 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-7fe65c20-fc77-4fd6-8aef-8ecb44cf7140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '341e65ec547b4ae5a911cc7d11127f16'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:11.477016 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1166/2332] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:27:11 2022] DELETE /identity/v3/users/341e65ec547b4ae5a911cc7d11127f16 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:11.484760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b952eeb-474f-4db3-b01d-b4b7e0080670 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:11.485265 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b952eeb-474f-4db3-b01d-b4b7e0080670 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:11.485379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b952eeb-474f-4db3-b01d-b4b7e0080670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:11.503639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5b952eeb-474f-4db3-b01d-b4b7e0080670 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:11.504894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5b952eeb-474f-4db3-b01d-b4b7e0080670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:11.579191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1167/2333] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:11.589517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe65c20-fc77-4fd6-8aef-8ecb44cf7140 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:11.590956 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-7fe65c20-fc77-4fd6-8aef-8ecb44cf7140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:11.658811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe65c20-fc77-4fd6-8aef-8ecb44cf7140 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:11.663880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2349c3-b469-4177-8ace-fac9f4426a31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:11.664735 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2349c3-b469-4177-8ace-fac9f4426a31 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:11.665019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2349c3-b469-4177-8ace-fac9f4426a31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:11.665292 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2349c3-b469-4177-8ace-fac9f4426a31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:11.676682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2349c3-b469-4177-8ace-fac9f4426a31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:11.678263 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2349c3-b469-4177-8ace-fac9f4426a31 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:11.689274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1167/2334] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:12.254453 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b952eeb-474f-4db3-b01d-b4b7e0080670 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:12.256881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5b952eeb-474f-4db3-b01d-b4b7e0080670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:12.287120 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b952eeb-474f-4db3-b01d-b4b7e0080670 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:12.296812 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e074f0-75c2-4bee-a140-fc03ab5c65ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:12.298860 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e074f0-75c2-4bee-a140-fc03ab5c65ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:12.299500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e074f0-75c2-4bee-a140-fc03ab5c65ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:12.300080 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e074f0-75c2-4bee-a140-fc03ab5c65ff admin admin] PATH_INFO: `/v3/projects/1d0e861e3eb6454eada1c1de4f5829bd` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:12.302705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e074f0-75c2-4bee-a140-fc03ab5c65ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d0e861e3eb6454eada1c1de4f5829bd)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:12.311016 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e074f0-75c2-4bee-a140-fc03ab5c65ff admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:12.406446 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-55e074f0-75c2-4bee-a140-fc03ab5c65ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d0e861e3eb6454eada1c1de4f5829bd', 'request_id': 'req-55e074f0-75c2-4bee-a140-fc03ab5c65ff'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:12.416007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-55e074f0-75c2-4bee-a140-fc03ab5c65ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d0e861e3eb6454eada1c1de4f5829bd', 'request_id': 'req-55e074f0-75c2-4bee-a140-fc03ab5c65ff'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:12.440076 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-55e074f0-75c2-4bee-a140-fc03ab5c65ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d0e861e3eb6454eada1c1de4f5829bd', 'request_id': 'req-55e074f0-75c2-4bee-a140-fc03ab5c65ff'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:12.457482 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-55e074f0-75c2-4bee-a140-fc03ab5c65ff admin admin] The token cache is being invalidate because project 1d0e861e3eb6454eada1c1de4f5829bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:27:12.458824 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-55e074f0-75c2-4bee-a140-fc03ab5c65ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:12.459996 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-55e074f0-75c2-4bee-a140-fc03ab5c65ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:12.462849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1168/2335] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:27:12 2022] DELETE /identity/v3/projects/1d0e861e3eb6454eada1c1de4f5829bd => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:27:12.472281 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83d9b0-0ccc-4254-b8d3-3c3af80e0d81 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:12.476071 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83d9b0-0ccc-4254-b8d3-3c3af80e0d81 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:12.476862 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83d9b0-0ccc-4254-b8d3-3c3af80e0d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:12.542572 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-2b83d9b0-0ccc-4254-b8d3-3c3af80e0d81 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:12.545773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-2b83d9b0-0ccc-4254-b8d3-3c3af80e0d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:12.573349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:12.625349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1168/2336] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:12.635424 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ec178-ec8c-4598-b46e-9bb3520011b6 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:12.636028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ec178-ec8c-4598-b46e-9bb3520011b6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:12.636853 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ec178-ec8c-4598-b46e-9bb3520011b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:12.678368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b72ec178-ec8c-4598-b46e-9bb3520011b6 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:12.679582 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b72ec178-ec8c-4598-b46e-9bb3520011b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:12.698851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1169/2337] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:12.699403 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83d9b0-0ccc-4254-b8d3-3c3af80e0d81 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:12.711807 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83d9b0-0ccc-4254-b8d3-3c3af80e0d81 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:12.718662 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9700a-20ae-4647-aa32-022a22f4acaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:12.719639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9700a-20ae-4647-aa32-022a22f4acaf admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:12.719996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9700a-20ae-4647-aa32-022a22f4acaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:12.720745 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9700a-20ae-4647-aa32-022a22f4acaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:12.721657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9700a-20ae-4647-aa32-022a22f4acaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:27:12.722049 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9700a-20ae-4647-aa32-022a22f4acaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:12.724217 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9700a-20ae-4647-aa32-022a22f4acaf admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:12.749924 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1169/2338] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:27:12.754254 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ec178-ec8c-4598-b46e-9bb3520011b6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:12.767675 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ec178-ec8c-4598-b46e-9bb3520011b6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:12.775379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ce04b-505b-4e18-8b25-a4d93a636c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:12.776644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ce04b-505b-4e18-8b25-a4d93a636c0d admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:12.777196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ce04b-505b-4e18-8b25-a4d93a636c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:12.777712 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ce04b-505b-4e18-8b25-a4d93a636c0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:12.779725 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ce04b-505b-4e18-8b25-a4d93a636c0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:12.782242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ce04b-505b-4e18-8b25-a4d93a636c0d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:12.843621 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1170/2339] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:12 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:12.847959 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9700a-20ae-4647-aa32-022a22f4acaf admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:12.860356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9700a-20ae-4647-aa32-022a22f4acaf admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:12.916974 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a81eb-7987-4612-9c54-3890c0fcdf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:12.917641 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a81eb-7987-4612-9c54-3890c0fcdf69 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:12.917920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a81eb-7987-4612-9c54-3890c0fcdf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:12.918194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a81eb-7987-4612-9c54-3890c0fcdf69 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:12.918891 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a81eb-7987-4612-9c54-3890c0fcdf69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:12.920840 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a81eb-7987-4612-9c54-3890c0fcdf69 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:12.941487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1170/2340] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:12 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:12.946531 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ce04b-505b-4e18-8b25-a4d93a636c0d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:12.956873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ce04b-505b-4e18-8b25-a4d93a636c0d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:12.961435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340faac-ed98-4d77-8d8a-9d0517d2a098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:12.962080 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340faac-ed98-4d77-8d8a-9d0517d2a098 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:12.962360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340faac-ed98-4d77-8d8a-9d0517d2a098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:12.962638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340faac-ed98-4d77-8d8a-9d0517d2a098 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:12.963366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340faac-ed98-4d77-8d8a-9d0517d2a098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:12.965026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340faac-ed98-4d77-8d8a-9d0517d2a098 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:12.975650 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1171/2341] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:12.979871 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a81eb-7987-4612-9c54-3890c0fcdf69 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:12.990054 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a81eb-7987-4612-9c54-3890c0fcdf69 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:12.995305 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea27a11-aad2-428d-b82a-4f9d447d5f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:12.995962 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea27a11-aad2-428d-b82a-4f9d447d5f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:12.996263 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea27a11-aad2-428d-b82a-4f9d447d5f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:12.996565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea27a11-aad2-428d-b82a-4f9d447d5f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:12.997316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea27a11-aad2-428d-b82a-4f9d447d5f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:12.998881 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea27a11-aad2-428d-b82a-4f9d447d5f53 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:13.009340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1171/2342] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:13.013466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340faac-ed98-4d77-8d8a-9d0517d2a098 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:13.023400 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340faac-ed98-4d77-8d8a-9d0517d2a098 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:13.028317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b382dfa-8500-4997-b014-874a79c60cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:13.029220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b382dfa-8500-4997-b014-874a79c60cf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:13.029505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b382dfa-8500-4997-b014-874a79c60cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:13.029784 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b382dfa-8500-4997-b014-874a79c60cf3 admin admin] PATH_INFO: `/v3/projects/1458cb408ee2432394cf011c258d9efa/users/3999dd4fffe0425b9ac0de59d6f5477d/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:13.046074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b382dfa-8500-4997-b014-874a79c60cf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1458cb408ee2432394cf011c258d9efa, user_id=3999dd4fffe0425b9ac0de59d6f5477d, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:13.048777 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b382dfa-8500-4997-b014-874a79c60cf3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:13.062360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1172/2343] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:13 2022] PUT /identity/v3/projects/1458cb408ee2432394cf011c258d9efa/users/3999dd4fffe0425b9ac0de59d6f5477d/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:13.073461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7603ec-2d9e-401b-8085-f9c8a976b191 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:13.073753 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7603ec-2d9e-401b-8085-f9c8a976b191 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:13.074033 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7603ec-2d9e-401b-8085-f9c8a976b191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:13.100990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4c7603ec-2d9e-401b-8085-f9c8a976b191 None None] MFA Rules not processed for user `3999dd4fffe0425b9ac0de59d6f5477d`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:13.102247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4c7603ec-2d9e-401b-8085-f9c8a976b191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:13.171201 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1172/2344] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:27:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:13.183242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b382dfa-8500-4997-b014-874a79c60cf3 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:13.184976 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2b382dfa-8500-4997-b014-874a79c60cf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:13.209826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:13.250903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b382dfa-8500-4997-b014-874a79c60cf3 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:13.256732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6c5b0-bf86-4a3d-ac52-122a72c0b4e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:13.257901 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6c5b0-bf86-4a3d-ac52-122a72c0b4e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:13.258183 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6c5b0-bf86-4a3d-ac52-122a72c0b4e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:13.258459 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6c5b0-bf86-4a3d-ac52-122a72c0b4e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:13.269630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6c5b0-bf86-4a3d-ac52-122a72c0b4e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:13.271077 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6c5b0-bf86-4a3d-ac52-122a72c0b4e4 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:13.283455 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1173/2345] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:13.309717 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7603ec-2d9e-401b-8085-f9c8a976b191 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:13.331203 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:13.396576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7603ec-2d9e-401b-8085-f9c8a976b191 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:13.402442 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e29671-b9e1-4334-af63-f0708afd674c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:13.403419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e29671-b9e1-4334-af63-f0708afd674c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:13.403705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e29671-b9e1-4334-af63-f0708afd674c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:13.403976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e29671-b9e1-4334-af63-f0708afd674c admin admin] PATH_INFO: `/v3/users/3999dd4fffe0425b9ac0de59d6f5477d` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:13.405279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e29671-b9e1-4334-af63-f0708afd674c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3999dd4fffe0425b9ac0de59d6f5477d)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:13.407260 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e29671-b9e1-4334-af63-f0708afd674c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:13.488509 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-61e29671-b9e1-4334-af63-f0708afd674c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3999dd4fffe0425b9ac0de59d6f5477d'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:13.497291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-61e29671-b9e1-4334-af63-f0708afd674c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3999dd4fffe0425b9ac0de59d6f5477d'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:13.512218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-61e29671-b9e1-4334-af63-f0708afd674c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3999dd4fffe0425b9ac0de59d6f5477d'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:13.519582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-61e29671-b9e1-4334-af63-f0708afd674c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3999dd4fffe0425b9ac0de59d6f5477d'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:13.520356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-61e29671-b9e1-4334-af63-f0708afd674c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3999dd4fffe0425b9ac0de59d6f5477d'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:13.528965 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1173/2346] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:27:13 2022] DELETE /identity/v3/users/3999dd4fffe0425b9ac0de59d6f5477d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:13.537808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4d22e-38b1-4f57-96db-3f83ca3ef8db None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:13.537808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4d22e-38b1-4f57-96db-3f83ca3ef8db None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:13.538074 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4d22e-38b1-4f57-96db-3f83ca3ef8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:13.556063 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-abc4d22e-38b1-4f57-96db-3f83ca3ef8db None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:13.557375 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-abc4d22e-38b1-4f57-96db-3f83ca3ef8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:13.617790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1174/2347] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:13.627328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e29671-b9e1-4334-af63-f0708afd674c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:13.628746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-61e29671-b9e1-4334-af63-f0708afd674c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:13.687183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e29671-b9e1-4334-af63-f0708afd674c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:13.691315 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee270c7-84c2-4f52-8d5f-c3e5f945e861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:13.692035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee270c7-84c2-4f52-8d5f-c3e5f945e861 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:13.692353 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee270c7-84c2-4f52-8d5f-c3e5f945e861 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:13.692633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee270c7-84c2-4f52-8d5f-c3e5f945e861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:13.702594 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee270c7-84c2-4f52-8d5f-c3e5f945e861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:13.704048 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee270c7-84c2-4f52-8d5f-c3e5f945e861 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:13.713675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1174/2348] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:14.075517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4d22e-38b1-4f57-96db-3f83ca3ef8db None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:14.077006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-abc4d22e-38b1-4f57-96db-3f83ca3ef8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:14.090250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4d22e-38b1-4f57-96db-3f83ca3ef8db None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:14.095036 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7a2b9-e2d8-4113-b4cd-96471c70d777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:14.095878 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7a2b9-e2d8-4113-b4cd-96471c70d777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:14.096181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7a2b9-e2d8-4113-b4cd-96471c70d777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:14.096476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7a2b9-e2d8-4113-b4cd-96471c70d777 admin admin] PATH_INFO: `/v3/projects/1458cb408ee2432394cf011c258d9efa` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:14.097672 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7a2b9-e2d8-4113-b4cd-96471c70d777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1458cb408ee2432394cf011c258d9efa)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:14.099600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7a2b9-e2d8-4113-b4cd-96471c70d777 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:14.146250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-23a7a2b9-e2d8-4113-b4cd-96471c70d777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1458cb408ee2432394cf011c258d9efa', 'request_id': 'req-23a7a2b9-e2d8-4113-b4cd-96471c70d777'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:14.152310 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-23a7a2b9-e2d8-4113-b4cd-96471c70d777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1458cb408ee2432394cf011c258d9efa', 'request_id': 'req-23a7a2b9-e2d8-4113-b4cd-96471c70d777'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:14.159416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-23a7a2b9-e2d8-4113-b4cd-96471c70d777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1458cb408ee2432394cf011c258d9efa', 'request_id': 'req-23a7a2b9-e2d8-4113-b4cd-96471c70d777'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:14.170525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-23a7a2b9-e2d8-4113-b4cd-96471c70d777 admin admin] The token cache is being invalidate because project 1458cb408ee2432394cf011c258d9efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:27:14.170838 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-23a7a2b9-e2d8-4113-b4cd-96471c70d777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:14.171498 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-23a7a2b9-e2d8-4113-b4cd-96471c70d777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:14.173143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1175/2349] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:27:14 2022] DELETE /identity/v3/projects/1458cb408ee2432394cf011c258d9efa => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:15.352996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c53ea6-d7e4-41fc-a704-d92e4f84d29b None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:15.353423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c53ea6-d7e4-41fc-a704-d92e4f84d29b None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:15.353549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c53ea6-d7e4-41fc-a704-d92e4f84d29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:15.373573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-c6c53ea6-d7e4-41fc-a704-d92e4f84d29b None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:15.374814 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c6c53ea6-d7e4-41fc-a704-d92e4f84d29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:15.411582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:15.435878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1175/2350] 10.222.0.43 () {58 vars in 967 bytes} [Wed Jul 20 19:27:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:15.460121 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7a2b9-e2d8-4113-b4cd-96471c70d777 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:15.461572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-23a7a2b9-e2d8-4113-b4cd-96471c70d777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:15.511606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:15.533726 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7a2b9-e2d8-4113-b4cd-96471c70d777 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:15.539379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b046a9-6502-43ed-afca-1f5deafdc4b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:15.540024 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b046a9-6502-43ed-afca-1f5deafdc4b4 service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:15.540329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b046a9-6502-43ed-afca-1f5deafdc4b4 service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:15.540606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b046a9-6502-43ed-afca-1f5deafdc4b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:15.548777 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b046a9-6502-43ed-afca-1f5deafdc4b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:15.550366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b046a9-6502-43ed-afca-1f5deafdc4b4 service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:15.559120 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1176/2351] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:27:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:27:15.651987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c53ea6-d7e4-41fc-a704-d92e4f84d29b None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:15.653461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c6c53ea6-d7e4-41fc-a704-d92e4f84d29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:15.726675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c53ea6-d7e4-41fc-a704-d92e4f84d29b None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:15.731283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff657a82-048a-48d4-87a5-716dc9909d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:15.732164 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff657a82-048a-48d4-87a5-716dc9909d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:15.732647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff657a82-048a-48d4-87a5-716dc9909d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:15.732887 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff657a82-048a-48d4-87a5-716dc9909d7f admin admin] PATH_INFO: `/v3/users/764c95c9af6c4b67b743b569b1696743` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:15.734213 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff657a82-048a-48d4-87a5-716dc9909d7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=764c95c9af6c4b67b743b569b1696743)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:15.736505 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff657a82-048a-48d4-87a5-716dc9909d7f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:15.784595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ff657a82-048a-48d4-87a5-716dc9909d7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '764c95c9af6c4b67b743b569b1696743'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:15.792604 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ff657a82-048a-48d4-87a5-716dc9909d7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '764c95c9af6c4b67b743b569b1696743'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:15.805863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ff657a82-048a-48d4-87a5-716dc9909d7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '764c95c9af6c4b67b743b569b1696743'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:15.812724 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ff657a82-048a-48d4-87a5-716dc9909d7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '764c95c9af6c4b67b743b569b1696743'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:15.813795 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ff657a82-048a-48d4-87a5-716dc9909d7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '764c95c9af6c4b67b743b569b1696743'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:15.820397 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1176/2352] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:27:15 2022] DELETE /identity/v3/users/764c95c9af6c4b67b743b569b1696743 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:15.824944 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77532e99-86f2-494c-bf4c-4f5f58692c7f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:15.825239 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77532e99-86f2-494c-bf4c-4f5f58692c7f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:15.825523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77532e99-86f2-494c-bf4c-4f5f58692c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:15.843704 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-77532e99-86f2-494c-bf4c-4f5f58692c7f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:15.844952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-77532e99-86f2-494c-bf4c-4f5f58692c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:15.904865 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1177/2353] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:27:15.915030 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff657a82-048a-48d4-87a5-716dc9909d7f admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:15.916528 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ff657a82-048a-48d4-87a5-716dc9909d7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:15.982075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff657a82-048a-48d4-87a5-716dc9909d7f admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:15.985872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7609e24-ab71-43ee-bb70-fa11b8123a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:15.986625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7609e24-ab71-43ee-bb70-fa11b8123a32 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:15.986809 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7609e24-ab71-43ee-bb70-fa11b8123a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:15.987086 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7609e24-ab71-43ee-bb70-fa11b8123a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:15.996881 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7609e24-ab71-43ee-bb70-fa11b8123a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:15.998332 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7609e24-ab71-43ee-bb70-fa11b8123a32 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:16.008383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1177/2354] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:16.309992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77532e99-86f2-494c-bf4c-4f5f58692c7f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:16.311429 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-77532e99-86f2-494c-bf4c-4f5f58692c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:16.324867 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77532e99-86f2-494c-bf4c-4f5f58692c7f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:16.331838 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ee53d-591f-460e-8b8b-dc3d696c149e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:16.332803 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ee53d-591f-460e-8b8b-dc3d696c149e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:16.336441 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ee53d-591f-460e-8b8b-dc3d696c149e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:16.336441 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ee53d-591f-460e-8b8b-dc3d696c149e admin admin] PATH_INFO: `/v3/projects/8e836e6c6faf4473941b42d5a978b88e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:16.337702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ee53d-591f-460e-8b8b-dc3d696c149e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e836e6c6faf4473941b42d5a978b88e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:16.343911 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ee53d-591f-460e-8b8b-dc3d696c149e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:16.411002 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ab5ee53d-591f-460e-8b8b-dc3d696c149e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e836e6c6faf4473941b42d5a978b88e', 'request_id': 'req-ab5ee53d-591f-460e-8b8b-dc3d696c149e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:16.418044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ab5ee53d-591f-460e-8b8b-dc3d696c149e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e836e6c6faf4473941b42d5a978b88e', 'request_id': 'req-ab5ee53d-591f-460e-8b8b-dc3d696c149e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:16.427025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ab5ee53d-591f-460e-8b8b-dc3d696c149e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e836e6c6faf4473941b42d5a978b88e', 'request_id': 'req-ab5ee53d-591f-460e-8b8b-dc3d696c149e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:16.444455 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ab5ee53d-591f-460e-8b8b-dc3d696c149e admin admin] The token cache is being invalidate because project 8e836e6c6faf4473941b42d5a978b88e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:27:16.444888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ab5ee53d-591f-460e-8b8b-dc3d696c149e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:16.445629 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ab5ee53d-591f-460e-8b8b-dc3d696c149e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:16.447357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1178/2355] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:27:16 2022] DELETE /identity/v3/projects/8e836e6c6faf4473941b42d5a978b88e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:21.327157 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89811fdc-94df-4e6c-a136-f79687c4d7ea None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:21.327657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89811fdc-94df-4e6c-a136-f79687c4d7ea None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:21.327752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89811fdc-94df-4e6c-a136-f79687c4d7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:21.340800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:21.347515 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-89811fdc-94df-4e6c-a136-f79687c4d7ea None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:21.349675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-89811fdc-94df-4e6c-a136-f79687c4d7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:21.412977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1178/2356] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jul 20 19:27:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:21.428406 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ee53d-591f-460e-8b8b-dc3d696c149e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:21.429897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ab5ee53d-591f-460e-8b8b-dc3d696c149e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:21.498923 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:21.507302 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ee53d-591f-460e-8b8b-dc3d696c149e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:21.512826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97493690-6db8-48e9-8f99-620894ce11a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:21.513480 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97493690-6db8-48e9-8f99-620894ce11a4 service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:21.513767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97493690-6db8-48e9-8f99-620894ce11a4 service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:21.514043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97493690-6db8-48e9-8f99-620894ce11a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:21.524154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97493690-6db8-48e9-8f99-620894ce11a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:21.525765 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97493690-6db8-48e9-8f99-620894ce11a4 service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:21.534425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1179/2357] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:27:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:27:22.041747 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dab89b-a728-419f-955b-693165cb9319 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:22.042043 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dab89b-a728-419f-955b-693165cb9319 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:22.042324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dab89b-a728-419f-955b-693165cb9319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:22.060080 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-b1dab89b-a728-419f-955b-693165cb9319 None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:22.061312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-b1dab89b-a728-419f-955b-693165cb9319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:22.069379 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1179/2358] 10.222.0.43 () {58 vars in 967 bytes} [Wed Jul 20 19:27:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:27:22.081577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97493690-6db8-48e9-8f99-620894ce11a4 service swift] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:22.091828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97493690-6db8-48e9-8f99-620894ce11a4 service swift] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:22.096838 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8439f49-66ea-446a-817a-7c163934477c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:22.097488 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8439f49-66ea-446a-817a-7c163934477c service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:22.097774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8439f49-66ea-446a-817a-7c163934477c service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:22.098048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8439f49-66ea-446a-817a-7c163934477c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:22.106028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8439f49-66ea-446a-817a-7c163934477c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:22.107591 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8439f49-66ea-446a-817a-7c163934477c service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:22.116401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1180/2359] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:27:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:27:22.306543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390e04b-5925-46f2-b636-8b22dd34d864 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:22.306841 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390e04b-5925-46f2-b636-8b22dd34d864 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:22.307115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390e04b-5925-46f2-b636-8b22dd34d864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:22.325048 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-c390e04b-5925-46f2-b636-8b22dd34d864 None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:22.326241 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c390e04b-5925-46f2-b636-8b22dd34d864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:22.333301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1180/2360] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jul 20 19:27:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:22.345658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8439f49-66ea-446a-817a-7c163934477c service swift] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:22.356003 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8439f49-66ea-446a-817a-7c163934477c service swift] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:22.361063 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe9683-5886-41f1-a691-978a705c7b2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:22.361797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe9683-5886-41f1-a691-978a705c7b2d service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:22.362069 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe9683-5886-41f1-a691-978a705c7b2d service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:22.362343 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe9683-5886-41f1-a691-978a705c7b2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:22.370622 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe9683-5886-41f1-a691-978a705c7b2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:22.372188 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe9683-5886-41f1-a691-978a705c7b2d service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:22.380395 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1181/2361] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:27:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:27:22.693739 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3faf6-583f-48d0-ba3a-3fc91747bf79 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:22.694024 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3faf6-583f-48d0-ba3a-3fc91747bf79 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:22.694299 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3faf6-583f-48d0-ba3a-3fc91747bf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:22.712265 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-1ac3faf6-583f-48d0-ba3a-3fc91747bf79 None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:22.713492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-1ac3faf6-583f-48d0-ba3a-3fc91747bf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:22.720712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1181/2362] 10.222.0.43 () {58 vars in 967 bytes} [Wed Jul 20 19:27:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:22.733281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe9683-5886-41f1-a691-978a705c7b2d service swift] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:22.744361 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe9683-5886-41f1-a691-978a705c7b2d service swift] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:22.749454 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f0a32-2db3-4fc8-a85a-04ac5ee71a95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:22.750089 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f0a32-2db3-4fc8-a85a-04ac5ee71a95 service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:22.750376 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f0a32-2db3-4fc8-a85a-04ac5ee71a95 service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:22.750654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f0a32-2db3-4fc8-a85a-04ac5ee71a95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:22.758951 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f0a32-2db3-4fc8-a85a-04ac5ee71a95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:22.760567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f0a32-2db3-4fc8-a85a-04ac5ee71a95 service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:22.768674 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1182/2363] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:27:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:27:24.865385 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3faf6-583f-48d0-ba3a-3fc91747bf79 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:24.866779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-1ac3faf6-583f-48d0-ba3a-3fc91747bf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:24.932606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3faf6-583f-48d0-ba3a-3fc91747bf79 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:24.937070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a689d-209d-45fd-8db5-5b1209a1b011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:24.937937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a689d-209d-45fd-8db5-5b1209a1b011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:24.938221 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a689d-209d-45fd-8db5-5b1209a1b011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:24.938756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a689d-209d-45fd-8db5-5b1209a1b011 admin admin] PATH_INFO: `/v3/users/a1ceda5bed824184b10bfaff48ee006c` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:24.939998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a689d-209d-45fd-8db5-5b1209a1b011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ceda5bed824184b10bfaff48ee006c)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:24.941938 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a689d-209d-45fd-8db5-5b1209a1b011 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:24.987699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c65a689d-209d-45fd-8db5-5b1209a1b011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ceda5bed824184b10bfaff48ee006c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:24.995318 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c65a689d-209d-45fd-8db5-5b1209a1b011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ceda5bed824184b10bfaff48ee006c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:25.007553 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c65a689d-209d-45fd-8db5-5b1209a1b011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ceda5bed824184b10bfaff48ee006c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:25.014230 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c65a689d-209d-45fd-8db5-5b1209a1b011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ceda5bed824184b10bfaff48ee006c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:25.014874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c65a689d-209d-45fd-8db5-5b1209a1b011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ceda5bed824184b10bfaff48ee006c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:25.020714 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1182/2364] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:27:24 2022] DELETE /identity/v3/users/a1ceda5bed824184b10bfaff48ee006c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:25.025336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463352c-4891-4007-b602-d3fc5aa607aa None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:25.025848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463352c-4891-4007-b602-d3fc5aa607aa None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:25.025942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463352c-4891-4007-b602-d3fc5aa607aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:25.043439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5463352c-4891-4007-b602-d3fc5aa607aa None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:25.044667 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5463352c-4891-4007-b602-d3fc5aa607aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:25.101790 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1183/2365] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:25.111678 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a689d-209d-45fd-8db5-5b1209a1b011 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:25.113109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c65a689d-209d-45fd-8db5-5b1209a1b011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:25.127593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:25.173777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a689d-209d-45fd-8db5-5b1209a1b011 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:25.177445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5b335-1ede-4a0e-b66f-2e8e91409a7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:25.178591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5b335-1ede-4a0e-b66f-2e8e91409a7f service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:25.178871 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5b335-1ede-4a0e-b66f-2e8e91409a7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:25.179147 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5b335-1ede-4a0e-b66f-2e8e91409a7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:25.188570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5b335-1ede-4a0e-b66f-2e8e91409a7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:25.189965 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5b335-1ede-4a0e-b66f-2e8e91409a7f service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:25.200044 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1183/2366] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:25.370052 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463352c-4891-4007-b602-d3fc5aa607aa None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:25.371487 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5463352c-4891-4007-b602-d3fc5aa607aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:25.385999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463352c-4891-4007-b602-d3fc5aa607aa None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:25.391064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2ed10-80ed-4371-97f4-9314523d24b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:25.391937 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2ed10-80ed-4371-97f4-9314523d24b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:25.392219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2ed10-80ed-4371-97f4-9314523d24b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:25.392524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2ed10-80ed-4371-97f4-9314523d24b2 admin admin] PATH_INFO: `/v3/projects/2765674134b8415f8ab79fc7295fbe1d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:25.394124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2ed10-80ed-4371-97f4-9314523d24b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2765674134b8415f8ab79fc7295fbe1d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:25.396036 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2ed10-80ed-4371-97f4-9314523d24b2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:25.443128 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2bd2ed10-80ed-4371-97f4-9314523d24b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2765674134b8415f8ab79fc7295fbe1d', 'request_id': 'req-2bd2ed10-80ed-4371-97f4-9314523d24b2'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:25.449174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2bd2ed10-80ed-4371-97f4-9314523d24b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2765674134b8415f8ab79fc7295fbe1d', 'request_id': 'req-2bd2ed10-80ed-4371-97f4-9314523d24b2'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:25.456450 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2bd2ed10-80ed-4371-97f4-9314523d24b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2765674134b8415f8ab79fc7295fbe1d', 'request_id': 'req-2bd2ed10-80ed-4371-97f4-9314523d24b2'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:25.468607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2bd2ed10-80ed-4371-97f4-9314523d24b2 admin admin] The token cache is being invalidate because project 2765674134b8415f8ab79fc7295fbe1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:27:25.468900 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2bd2ed10-80ed-4371-97f4-9314523d24b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:25.469465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2bd2ed10-80ed-4371-97f4-9314523d24b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:25.471244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1184/2367] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:27:25 2022] DELETE /identity/v3/projects/2765674134b8415f8ab79fc7295fbe1d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:25.478903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e100284-990c-4b56-9a7f-ec680f8c6b8c None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:25.479189 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e100284-990c-4b56-9a7f-ec680f8c6b8c None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:25.479785 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e100284-990c-4b56-9a7f-ec680f8c6b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:25.497488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-6e100284-990c-4b56-9a7f-ec680f8c6b8c None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:25.498689 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6e100284-990c-4b56-9a7f-ec680f8c6b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:25.556236 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1184/2368] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:25.618888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff9845-a5f0-432f-959a-a2ee51a5813f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:25.619098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff9845-a5f0-432f-959a-a2ee51a5813f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:25.619422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff9845-a5f0-432f-959a-a2ee51a5813f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:25.637555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-90ff9845-a5f0-432f-959a-a2ee51a5813f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:25.638758 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-90ff9845-a5f0-432f-959a-a2ee51a5813f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:25.646744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1185/2369] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:25.651133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e100284-990c-4b56-9a7f-ec680f8c6b8c None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:25.660768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e100284-990c-4b56-9a7f-ec680f8c6b8c None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:25.665806 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b41f7b-d452-4944-a291-029a8fbc3d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:25.666454 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b41f7b-d452-4944-a291-029a8fbc3d91 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:25.666737 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b41f7b-d452-4944-a291-029a8fbc3d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:25.667013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b41f7b-d452-4944-a291-029a8fbc3d91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:25.667761 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b41f7b-d452-4944-a291-029a8fbc3d91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:27:25.668415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b41f7b-d452-4944-a291-029a8fbc3d91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:25.669690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b41f7b-d452-4944-a291-029a8fbc3d91 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:25.681619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1185/2370] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:27:25.685883 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff9845-a5f0-432f-959a-a2ee51a5813f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:25.695385 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff9845-a5f0-432f-959a-a2ee51a5813f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:25.700058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df4ecc-ac0d-45e6-97fc-43f6dfd583fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:25.700744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df4ecc-ac0d-45e6-97fc-43f6dfd583fc admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:25.701035 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df4ecc-ac0d-45e6-97fc-43f6dfd583fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:25.701313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df4ecc-ac0d-45e6-97fc-43f6dfd583fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:25.702005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df4ecc-ac0d-45e6-97fc-43f6dfd583fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:25.703900 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df4ecc-ac0d-45e6-97fc-43f6dfd583fc admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:25.730563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1186/2371] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:25 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:25.734859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b41f7b-d452-4944-a291-029a8fbc3d91 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:25.745022 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b41f7b-d452-4944-a291-029a8fbc3d91 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:25.801520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df2fbe-b8a5-4def-9375-07f5122adf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:25.802191 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df2fbe-b8a5-4def-9375-07f5122adf49 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:25.802478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df2fbe-b8a5-4def-9375-07f5122adf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:25.802760 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df2fbe-b8a5-4def-9375-07f5122adf49 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:25.803497 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df2fbe-b8a5-4def-9375-07f5122adf49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:25.805451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df2fbe-b8a5-4def-9375-07f5122adf49 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:25.824394 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1186/2372] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:25 2022] POST /identity/v3/users => generated 300 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:25.828382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df4ecc-ac0d-45e6-97fc-43f6dfd583fc admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:25.835505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:25.839432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df4ecc-ac0d-45e6-97fc-43f6dfd583fc admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:25.845350 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c812ee-6518-4130-8425-a56e06681778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:25.846014 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c812ee-6518-4130-8425-a56e06681778 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:25.846480 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c812ee-6518-4130-8425-a56e06681778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:25.846916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c812ee-6518-4130-8425-a56e06681778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:25.847856 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c812ee-6518-4130-8425-a56e06681778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:25.849494 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c812ee-6518-4130-8425-a56e06681778 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:25.862857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1187/2373] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:25.866959 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df2fbe-b8a5-4def-9375-07f5122adf49 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:25.877079 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df2fbe-b8a5-4def-9375-07f5122adf49 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:25.881848 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b601719-e1f5-4f69-9778-49fba00d84e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:25.882506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b601719-e1f5-4f69-9778-49fba00d84e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:25.882818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b601719-e1f5-4f69-9778-49fba00d84e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:25.883094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b601719-e1f5-4f69-9778-49fba00d84e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:25.883816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b601719-e1f5-4f69-9778-49fba00d84e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:25.885421 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b601719-e1f5-4f69-9778-49fba00d84e7 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:25.895321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1187/2374] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:25.900185 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c812ee-6518-4130-8425-a56e06681778 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:25.910201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c812ee-6518-4130-8425-a56e06681778 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:25.915594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e840179-b061-47c5-be01-1504dd5e4746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:25.916537 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e840179-b061-47c5-be01-1504dd5e4746 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:25.916821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e840179-b061-47c5-be01-1504dd5e4746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:25.917097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e840179-b061-47c5-be01-1504dd5e4746 admin admin] PATH_INFO: `/v3/projects/3ee2f1db45854c0aae30345c85b4be73/users/d83b45e72eb44899bf1c11dccb2a13de/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:25.932893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e840179-b061-47c5-be01-1504dd5e4746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee2f1db45854c0aae30345c85b4be73, user_id=d83b45e72eb44899bf1c11dccb2a13de, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:25.935530 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e840179-b061-47c5-be01-1504dd5e4746 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:25.949663 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1188/2375] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:25 2022] PUT /identity/v3/projects/3ee2f1db45854c0aae30345c85b4be73/users/d83b45e72eb44899bf1c11dccb2a13de/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:25.959685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80014a4-3016-45fe-a245-b2c7db3b5b33 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:25.959974 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80014a4-3016-45fe-a245-b2c7db3b5b33 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:25.960253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80014a4-3016-45fe-a245-b2c7db3b5b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:25.986734 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-f80014a4-3016-45fe-a245-b2c7db3b5b33 None None] MFA Rules not processed for user `d83b45e72eb44899bf1c11dccb2a13de`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:25.987939 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f80014a4-3016-45fe-a245-b2c7db3b5b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:25.996832 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:26.054401 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1188/2376] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:27:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:26.066242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e840179-b061-47c5-be01-1504dd5e4746 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:26.067973 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4e840179-b061-47c5-be01-1504dd5e4746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:26.127306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e840179-b061-47c5-be01-1504dd5e4746 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:26.130833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac307c-6297-45b9-adfb-6780a733c4d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:26.131497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac307c-6297-45b9-adfb-6780a733c4d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:26.131782 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac307c-6297-45b9-adfb-6780a733c4d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:26.132060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac307c-6297-45b9-adfb-6780a733c4d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:26.140374 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac307c-6297-45b9-adfb-6780a733c4d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:26.141808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac307c-6297-45b9-adfb-6780a733c4d5 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:26.150539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1189/2377] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:34.235517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80014a4-3016-45fe-a245-b2c7db3b5b33 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:34.297511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80014a4-3016-45fe-a245-b2c7db3b5b33 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:34.302046 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d894d-fc90-40d5-828c-e27fdc5c694d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:34.302941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d894d-fc90-40d5-828c-e27fdc5c694d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:34.303230 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d894d-fc90-40d5-828c-e27fdc5c694d admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:34.303507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d894d-fc90-40d5-828c-e27fdc5c694d admin admin] PATH_INFO: `/v3/users/d83b45e72eb44899bf1c11dccb2a13de` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:34.304756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d894d-fc90-40d5-828c-e27fdc5c694d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d83b45e72eb44899bf1c11dccb2a13de)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:34.306664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d894d-fc90-40d5-828c-e27fdc5c694d admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:34.356349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-fd6d894d-fc90-40d5-828c-e27fdc5c694d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd83b45e72eb44899bf1c11dccb2a13de'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:34.364525 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-fd6d894d-fc90-40d5-828c-e27fdc5c694d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd83b45e72eb44899bf1c11dccb2a13de'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:34.379565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-fd6d894d-fc90-40d5-828c-e27fdc5c694d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd83b45e72eb44899bf1c11dccb2a13de'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:34.387006 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-fd6d894d-fc90-40d5-828c-e27fdc5c694d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd83b45e72eb44899bf1c11dccb2a13de'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:34.387718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-fd6d894d-fc90-40d5-828c-e27fdc5c694d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd83b45e72eb44899bf1c11dccb2a13de'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:34.394723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1189/2378] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:27:34 2022] DELETE /identity/v3/users/d83b45e72eb44899bf1c11dccb2a13de => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:34.400390 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a12dd-ab56-4f68-becf-b2cca2ef969a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:34.400903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a12dd-ab56-4f68-becf-b2cca2ef969a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:34.401102 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a12dd-ab56-4f68-becf-b2cca2ef969a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:34.430957 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d31a12dd-ab56-4f68-becf-b2cca2ef969a None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:34.432215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d31a12dd-ab56-4f68-becf-b2cca2ef969a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:34.495402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1190/2379] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:27:34.504996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d894d-fc90-40d5-828c-e27fdc5c694d admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:34.506448 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-fd6d894d-fc90-40d5-828c-e27fdc5c694d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:34.538196 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:34.564650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d894d-fc90-40d5-828c-e27fdc5c694d admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:34.568311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897571d-6c7e-43b2-a408-46f065f8b1da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:34.569599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897571d-6c7e-43b2-a408-46f065f8b1da service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:34.569880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897571d-6c7e-43b2-a408-46f065f8b1da service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:34.570153 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897571d-6c7e-43b2-a408-46f065f8b1da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:34.579232 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897571d-6c7e-43b2-a408-46f065f8b1da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:34.580751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897571d-6c7e-43b2-a408-46f065f8b1da service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:34.591778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1190/2380] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:34.865295 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a12dd-ab56-4f68-becf-b2cca2ef969a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:34.867548 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d31a12dd-ab56-4f68-becf-b2cca2ef969a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:34.882490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a12dd-ab56-4f68-becf-b2cca2ef969a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:34.887182 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe674d8b-1b38-414a-a4d9-933bb1671fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:34.888059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe674d8b-1b38-414a-a4d9-933bb1671fa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:34.888361 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe674d8b-1b38-414a-a4d9-933bb1671fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:34.888641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe674d8b-1b38-414a-a4d9-933bb1671fa0 admin admin] PATH_INFO: `/v3/projects/3ee2f1db45854c0aae30345c85b4be73` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:34.890649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe674d8b-1b38-414a-a4d9-933bb1671fa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee2f1db45854c0aae30345c85b4be73)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:34.893117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe674d8b-1b38-414a-a4d9-933bb1671fa0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:34.941297 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fe674d8b-1b38-414a-a4d9-933bb1671fa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee2f1db45854c0aae30345c85b4be73', 'request_id': 'req-fe674d8b-1b38-414a-a4d9-933bb1671fa0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:34.947433 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fe674d8b-1b38-414a-a4d9-933bb1671fa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee2f1db45854c0aae30345c85b4be73', 'request_id': 'req-fe674d8b-1b38-414a-a4d9-933bb1671fa0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:34.955265 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fe674d8b-1b38-414a-a4d9-933bb1671fa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee2f1db45854c0aae30345c85b4be73', 'request_id': 'req-fe674d8b-1b38-414a-a4d9-933bb1671fa0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:34.960487 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:34.968190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fe674d8b-1b38-414a-a4d9-933bb1671fa0 admin admin] The token cache is being invalidate because project 3ee2f1db45854c0aae30345c85b4be73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:27:34.968561 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fe674d8b-1b38-414a-a4d9-933bb1671fa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:34.969549 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fe674d8b-1b38-414a-a4d9-933bb1671fa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:34.971343 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1191/2381] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:27:34 2022] DELETE /identity/v3/projects/3ee2f1db45854c0aae30345c85b4be73 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:44.587152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=13 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:44.973422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:04.012180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbba27-5c06-4bf4-ae3f-6cc7b9ee264f None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:04.012729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbba27-5c06-4bf4-ae3f-6cc7b9ee264f None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:04.012729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbba27-5c06-4bf4-ae3f-6cc7b9ee264f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:04.029378 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-1ebbba27-5c06-4bf4-ae3f-6cc7b9ee264f None None] MFA Rules not processed for user `9ac790383bdf4eecaef96cc25fe38553`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:04.030619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-1ebbba27-5c06-4bf4-ae3f-6cc7b9ee264f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:04.084761 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1191/2382] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:28:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:12.567384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd88f7-45da-43a6-a7c3-10bfb9fff912 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:12.567916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd88f7-45da-43a6-a7c3-10bfb9fff912 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:12.568015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd88f7-45da-43a6-a7c3-10bfb9fff912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:12.585614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-45bd88f7-45da-43a6-a7c3-10bfb9fff912 None None] MFA Rules not processed for user `cb06eeba738b452fab3986de7e1f7856`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:12.586833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-45bd88f7-45da-43a6-a7c3-10bfb9fff912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:12.641952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1192/2383] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:28:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:14.077878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=25 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:22.635168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=25 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:31.235131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d9ecb-638c-44ff-a2a4-693434935d13 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:31.235587 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d9ecb-638c-44ff-a2a4-693434935d13 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:31.235697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d9ecb-638c-44ff-a2a4-693434935d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:31.253620 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-105d9ecb-638c-44ff-a2a4-693434935d13 None None] MFA Rules not processed for user `b15f8d66d5104aaeae51e6c1fa3b3596`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:31.254826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-105d9ecb-638c-44ff-a2a4-693434935d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:31.309613 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1192/2384] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:28:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:31.700501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f843b2b-6160-4e55-8a15-903eb5a09aa6 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:31.701046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f843b2b-6160-4e55-8a15-903eb5a09aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:31.701046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f843b2b-6160-4e55-8a15-903eb5a09aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:31.720301 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1f843b2b-6160-4e55-8a15-903eb5a09aa6 None None] MFA Rules not processed for user `4a4f4d585adf40b880be44c29ecd53cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:31.721611 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1f843b2b-6160-4e55-8a15-903eb5a09aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:31.777645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1193/2385] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:28:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:41.301895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=25 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:41.770290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=25 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:56.417401 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d9ecb-638c-44ff-a2a4-693434935d13 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:56.418856 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-105d9ecb-638c-44ff-a2a4-693434935d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:56.479632 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d9ecb-638c-44ff-a2a4-693434935d13 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:56.483469 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae6bd8-3e78-4160-be5f-8c293b182ab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:56.484115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae6bd8-3e78-4160-be5f-8c293b182ab1 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:56.484431 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae6bd8-3e78-4160-be5f-8c293b182ab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:56.484709 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae6bd8-3e78-4160-be5f-8c293b182ab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:56.486481 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-7aae6bd8-3e78-4160-be5f-8c293b182ab1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:56.499043 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae6bd8-3e78-4160-be5f-8c293b182ab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:56.500518 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae6bd8-3e78-4160-be5f-8c293b182ab1 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:56.509737 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1193/2386] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:56.617131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f843b2b-6160-4e55-8a15-903eb5a09aa6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:56.618558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1f843b2b-6160-4e55-8a15-903eb5a09aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:56.631284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f843b2b-6160-4e55-8a15-903eb5a09aa6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:56.635147 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2a6b8-1a42-488b-8245-83f61ee1a8c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:56.636423 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2a6b8-1a42-488b-8245-83f61ee1a8c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:56.636707 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2a6b8-1a42-488b-8245-83f61ee1a8c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:56.636979 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2a6b8-1a42-488b-8245-83f61ee1a8c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:56.638830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a3e2a6b8-1a42-488b-8245-83f61ee1a8c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:56.714662 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2a6b8-1a42-488b-8245-83f61ee1a8c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:56.716145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2a6b8-1a42-488b-8245-83f61ee1a8c8 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:56.728001 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1194/2387] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:00.549683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae6bd8-3e78-4160-be5f-8c293b182ab1 service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:00.551274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-7aae6bd8-3e78-4160-be5f-8c293b182ab1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:00.609408 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae6bd8-3e78-4160-be5f-8c293b182ab1 service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:00.613267 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e11485-e572-467e-9d0e-f12dd91bcbb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:00.613940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e11485-e572-467e-9d0e-f12dd91bcbb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:00.614313 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e11485-e572-467e-9d0e-f12dd91bcbb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:00.614592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e11485-e572-467e-9d0e-f12dd91bcbb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:00.616546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-79e11485-e572-467e-9d0e-f12dd91bcbb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:00.631287 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e11485-e572-467e-9d0e-f12dd91bcbb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:00.632981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e11485-e572-467e-9d0e-f12dd91bcbb0 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:00.646231 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1194/2388] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:04.621461 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2a6b8-1a42-488b-8245-83f61ee1a8c8 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:04.622977 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a3e2a6b8-1a42-488b-8245-83f61ee1a8c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:04.636943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2a6b8-1a42-488b-8245-83f61ee1a8c8 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:04.642609 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630a054-55d2-4bf1-946f-ff3bd2a2058c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:04.643268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630a054-55d2-4bf1-946f-ff3bd2a2058c service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:04.643555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630a054-55d2-4bf1-946f-ff3bd2a2058c service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:04.643838 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630a054-55d2-4bf1-946f-ff3bd2a2058c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:04.645825 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8630a054-55d2-4bf1-946f-ff3bd2a2058c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:04.657999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630a054-55d2-4bf1-946f-ff3bd2a2058c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:04.659619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630a054-55d2-4bf1-946f-ff3bd2a2058c service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:04.670765 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1195/2389] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:10.641196 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=52 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:14.664941 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=40 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:18.873448 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e11485-e572-467e-9d0e-f12dd91bcbb0 service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:18.882043 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e11485-e572-467e-9d0e-f12dd91bcbb0 service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:18.885842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac29d92-a6be-4964-a3e9-8621bc9b5623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:18.886475 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac29d92-a6be-4964-a3e9-8621bc9b5623 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:18.886755 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac29d92-a6be-4964-a3e9-8621bc9b5623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:18.887026 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac29d92-a6be-4964-a3e9-8621bc9b5623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:18.888820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4ac29d92-a6be-4964-a3e9-8621bc9b5623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:18.951848 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac29d92-a6be-4964-a3e9-8621bc9b5623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:18.953310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac29d92-a6be-4964-a3e9-8621bc9b5623 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:18.963023 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1195/2390] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:22.169105 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630a054-55d2-4bf1-946f-ff3bd2a2058c service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:22.170483 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8630a054-55d2-4bf1-946f-ff3bd2a2058c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:22.238187 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac29d92-a6be-4964-a3e9-8621bc9b5623 service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:22.240202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4ac29d92-a6be-4964-a3e9-8621bc9b5623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:22.253463 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630a054-55d2-4bf1-946f-ff3bd2a2058c service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:22.253663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac29d92-a6be-4964-a3e9-8621bc9b5623 service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:22.258234 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e0869-9050-4eb7-962c-da4b0a5725d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:22.258699 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32355a9a-d08a-4613-8206-7d41768185cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae581d3dda354e07894cb5205d1b2c5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:22.259101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e0869-9050-4eb7-962c-da4b0a5725d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.259339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32355a9a-d08a-4613-8206-7d41768185cc service placement] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.259448 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e0869-9050-4eb7-962c-da4b0a5725d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.259626 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32355a9a-d08a-4613-8206-7d41768185cc service placement] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.259749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e0869-9050-4eb7-962c-da4b0a5725d9 admin admin] PATH_INFO: `/v3/users/cb06eeba738b452fab3986de7e1f7856` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.259905 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32355a9a-d08a-4613-8206-7d41768185cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.260949 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e0869-9050-4eb7-962c-da4b0a5725d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb06eeba738b452fab3986de7e1f7856)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:22.261848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-32355a9a-d08a-4613-8206-7d41768185cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:22.263010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e0869-9050-4eb7-962c-da4b0a5725d9 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:22.273746 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32355a9a-d08a-4613-8206-7d41768185cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:22.275430 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32355a9a-d08a-4613-8206-7d41768185cc service placement] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:22.286260 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1196/2391] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:22.318689 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-036e0869-9050-4eb7-962c-da4b0a5725d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb06eeba738b452fab3986de7e1f7856'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:22.326412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-036e0869-9050-4eb7-962c-da4b0a5725d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb06eeba738b452fab3986de7e1f7856'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:22.338318 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-036e0869-9050-4eb7-962c-da4b0a5725d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb06eeba738b452fab3986de7e1f7856'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:22.346002 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-036e0869-9050-4eb7-962c-da4b0a5725d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb06eeba738b452fab3986de7e1f7856'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:22.346861 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-036e0869-9050-4eb7-962c-da4b0a5725d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb06eeba738b452fab3986de7e1f7856'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:22.353088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1196/2392] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:22 2022] DELETE /identity/v3/users/cb06eeba738b452fab3986de7e1f7856 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:22.523542 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32355a9a-d08a-4613-8206-7d41768185cc service placement] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:22.537967 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32355a9a-d08a-4613-8206-7d41768185cc service placement] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:22.597384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a9f8c-66d0-40ca-ba3a-5bbbcaa17e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:22.598244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a9f8c-66d0-40ca-ba3a-5bbbcaa17e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.598528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a9f8c-66d0-40ca-ba3a-5bbbcaa17e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.598808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a9f8c-66d0-40ca-ba3a-5bbbcaa17e6a admin admin] PATH_INFO: `/v3/projects/f0f63357734c4efa9a88c07996aa5551` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.599980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a9f8c-66d0-40ca-ba3a-5bbbcaa17e6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0f63357734c4efa9a88c07996aa5551)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:22.601946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a9f8c-66d0-40ca-ba3a-5bbbcaa17e6a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:22.647953 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f11a9f8c-66d0-40ca-ba3a-5bbbcaa17e6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0f63357734c4efa9a88c07996aa5551', 'request_id': 'req-f11a9f8c-66d0-40ca-ba3a-5bbbcaa17e6a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:22.654147 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f11a9f8c-66d0-40ca-ba3a-5bbbcaa17e6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0f63357734c4efa9a88c07996aa5551', 'request_id': 'req-f11a9f8c-66d0-40ca-ba3a-5bbbcaa17e6a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:22.661894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f11a9f8c-66d0-40ca-ba3a-5bbbcaa17e6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0f63357734c4efa9a88c07996aa5551', 'request_id': 'req-f11a9f8c-66d0-40ca-ba3a-5bbbcaa17e6a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:22.672312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f11a9f8c-66d0-40ca-ba3a-5bbbcaa17e6a admin admin] The token cache is being invalidate because project f0f63357734c4efa9a88c07996aa5551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:22.672609 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f11a9f8c-66d0-40ca-ba3a-5bbbcaa17e6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:22.673451 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f11a9f8c-66d0-40ca-ba3a-5bbbcaa17e6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:22.675467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1197/2393] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:22 2022] DELETE /identity/v3/projects/f0f63357734c4efa9a88c07996aa5551 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:22.680518 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194e3a7-c04f-47a1-a70f-736addc0fc38 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.680811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194e3a7-c04f-47a1-a70f-736addc0fc38 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.681091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194e3a7-c04f-47a1-a70f-736addc0fc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.699842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-0194e3a7-c04f-47a1-a70f-736addc0fc38 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:22.701085 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0194e3a7-c04f-47a1-a70f-736addc0fc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:22.762309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1197/2394] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:22.769715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680a95b-9156-4940-8af7-d3928e281c85 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.770000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680a95b-9156-4940-8af7-d3928e281c85 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.770274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680a95b-9156-4940-8af7-d3928e281c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.790827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8680a95b-9156-4940-8af7-d3928e281c85 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:22.792188 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8680a95b-9156-4940-8af7-d3928e281c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:22.800850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1198/2395] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:22.805118 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194e3a7-c04f-47a1-a70f-736addc0fc38 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:22.815820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194e3a7-c04f-47a1-a70f-736addc0fc38 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:22.820707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7694b4e-cdc1-413e-b5f8-da0d0c56d024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:22.821383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7694b4e-cdc1-413e-b5f8-da0d0c56d024 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.821645 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7694b4e-cdc1-413e-b5f8-da0d0c56d024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.821952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7694b4e-cdc1-413e-b5f8-da0d0c56d024 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.822683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7694b4e-cdc1-413e-b5f8-da0d0c56d024 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:29:22.823006 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7694b4e-cdc1-413e-b5f8-da0d0c56d024 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:22.825218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7694b4e-cdc1-413e-b5f8-da0d0c56d024 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:22.826814 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:22.841721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1198/2396] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:29:22.842853 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680a95b-9156-4940-8af7-d3928e281c85 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:22.851969 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680a95b-9156-4940-8af7-d3928e281c85 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:22.856956 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2a6c6-0f87-46fa-a92e-c1b05cce9fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:22.857594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2a6c6-0f87-46fa-a92e-c1b05cce9fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.857872 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2a6c6-0f87-46fa-a92e-c1b05cce9fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.858141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2a6c6-0f87-46fa-a92e-c1b05cce9fa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.858841 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2a6c6-0f87-46fa-a92e-c1b05cce9fa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:22.860767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2a6c6-0f87-46fa-a92e-c1b05cce9fa8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:22.870487 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:22.894214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1199/2397] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:22 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:29:22.895352 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7694b4e-cdc1-413e-b5f8-da0d0c56d024 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:22.905398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7694b4e-cdc1-413e-b5f8-da0d0c56d024 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:22.964901 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379c842-a989-4f33-b35f-12cfa15de4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:22.965660 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379c842-a989-4f33-b35f-12cfa15de4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.966023 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379c842-a989-4f33-b35f-12cfa15de4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.966360 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379c842-a989-4f33-b35f-12cfa15de4a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.967189 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c379c842-a989-4f33-b35f-12cfa15de4a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:22.969590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c379c842-a989-4f33-b35f-12cfa15de4a2 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:22.999998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1199/2398] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:22 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:23.001608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2a6c6-0f87-46fa-a92e-c1b05cce9fa8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:23.012336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2a6c6-0f87-46fa-a92e-c1b05cce9fa8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:23.017609 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04507df-6e70-4e14-9923-b9a7e9e789d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:23.018202 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04507df-6e70-4e14-9923-b9a7e9e789d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:23.018491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04507df-6e70-4e14-9923-b9a7e9e789d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:23.018769 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04507df-6e70-4e14-9923-b9a7e9e789d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:23.019683 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04507df-6e70-4e14-9923-b9a7e9e789d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:23.021321 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04507df-6e70-4e14-9923-b9a7e9e789d4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:23.034883 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1200/2399] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:23.038897 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379c842-a989-4f33-b35f-12cfa15de4a2 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:23.050427 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379c842-a989-4f33-b35f-12cfa15de4a2 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:23.055293 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc876b-751a-4e27-bfa9-9ba9125adf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:23.056685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc876b-751a-4e27-bfa9-9ba9125adf28 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:23.056685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc876b-751a-4e27-bfa9-9ba9125adf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:23.056886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc876b-751a-4e27-bfa9-9ba9125adf28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:23.057569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc876b-751a-4e27-bfa9-9ba9125adf28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:23.059757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc876b-751a-4e27-bfa9-9ba9125adf28 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:23.070351 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1200/2400] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:29:23.074654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04507df-6e70-4e14-9923-b9a7e9e789d4 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:23.084843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04507df-6e70-4e14-9923-b9a7e9e789d4 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:23.089532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c542e6-9fe8-4b9e-976b-678589ccc4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:23.090471 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c542e6-9fe8-4b9e-976b-678589ccc4f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:23.091185 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c542e6-9fe8-4b9e-976b-678589ccc4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:23.091185 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c542e6-9fe8-4b9e-976b-678589ccc4f5 admin admin] PATH_INFO: `/v3/projects/b5a8676f5a1349b08a7741bb53838d16/users/a0c0becb185f49ff86242fe80a0394c3/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:23.106071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c542e6-9fe8-4b9e-976b-678589ccc4f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5a8676f5a1349b08a7741bb53838d16, user_id=a0c0becb185f49ff86242fe80a0394c3, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:23.108710 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c542e6-9fe8-4b9e-976b-678589ccc4f5 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:23.123413 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1201/2401] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:23 2022] PUT /identity/v3/projects/b5a8676f5a1349b08a7741bb53838d16/users/a0c0becb185f49ff86242fe80a0394c3/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:23.134166 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e7321-0974-43aa-a57e-de7f10d82d9c None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:23.134461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e7321-0974-43aa-a57e-de7f10d82d9c None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:23.134740 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e7321-0974-43aa-a57e-de7f10d82d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:23.160818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-d29e7321-0974-43aa-a57e-de7f10d82d9c None None] MFA Rules not processed for user `a0c0becb185f49ff86242fe80a0394c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:23.161904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d29e7321-0974-43aa-a57e-de7f10d82d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:23.224835 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1201/2402] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:29:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:23.235906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c542e6-9fe8-4b9e-976b-678589ccc4f5 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:23.237627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b5c542e6-9fe8-4b9e-976b-678589ccc4f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:23.309993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c542e6-9fe8-4b9e-976b-678589ccc4f5 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:23.315758 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400091f-f40c-4d70-94ec-ecb908ef84ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:23.316436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400091f-f40c-4d70-94ec-ecb908ef84ed service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:23.316726 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400091f-f40c-4d70-94ec-ecb908ef84ed service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:23.317008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400091f-f40c-4d70-94ec-ecb908ef84ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:23.325130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400091f-f40c-4d70-94ec-ecb908ef84ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:23.326721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400091f-f40c-4d70-94ec-ecb908ef84ed service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:23.344686 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1202/2403] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:25.426716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cbdad-9347-4cce-9400-f101a1670294 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:25.427212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cbdad-9347-4cce-9400-f101a1670294 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:25.427412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cbdad-9347-4cce-9400-f101a1670294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:25.446006 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-f95cbdad-9347-4cce-9400-f101a1670294 None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:25.447330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f95cbdad-9347-4cce-9400-f101a1670294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:25.504246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1202/2404] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jul 20 19:29:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:25.550065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400091f-f40c-4d70-94ec-ecb908ef84ed service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:25.551519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5400091f-f40c-4d70-94ec-ecb908ef84ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:25.621149 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400091f-f40c-4d70-94ec-ecb908ef84ed service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:25.626142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec84d1-a790-4e28-9032-f5afd4b84ed9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:25.626802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec84d1-a790-4e28-9032-f5afd4b84ed9 service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:25.627086 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec84d1-a790-4e28-9032-f5afd4b84ed9 service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:25.627364 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec84d1-a790-4e28-9032-f5afd4b84ed9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:25.635692 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec84d1-a790-4e28-9032-f5afd4b84ed9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:25.637308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec84d1-a790-4e28-9032-f5afd4b84ed9 service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:25.645457 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1203/2405] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:29:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:29:26.002710 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cbdad-9347-4cce-9400-f101a1670294 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:26.013532 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:26.079488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cbdad-9347-4cce-9400-f101a1670294 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:26.084040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232105e0-9ffd-473a-bb7e-458c0fcf736b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:26.084946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232105e0-9ffd-473a-bb7e-458c0fcf736b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:26.085226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232105e0-9ffd-473a-bb7e-458c0fcf736b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:26.085509 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232105e0-9ffd-473a-bb7e-458c0fcf736b admin admin] PATH_INFO: `/v3/users/a0c0becb185f49ff86242fe80a0394c3` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:26.086699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232105e0-9ffd-473a-bb7e-458c0fcf736b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0c0becb185f49ff86242fe80a0394c3)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:26.094625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232105e0-9ffd-473a-bb7e-458c0fcf736b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:26.151035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-232105e0-9ffd-473a-bb7e-458c0fcf736b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0c0becb185f49ff86242fe80a0394c3'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:26.161864 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-232105e0-9ffd-473a-bb7e-458c0fcf736b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0c0becb185f49ff86242fe80a0394c3'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:26.179553 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-232105e0-9ffd-473a-bb7e-458c0fcf736b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0c0becb185f49ff86242fe80a0394c3'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:26.190407 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-232105e0-9ffd-473a-bb7e-458c0fcf736b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0c0becb185f49ff86242fe80a0394c3'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:26.191261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-232105e0-9ffd-473a-bb7e-458c0fcf736b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0c0becb185f49ff86242fe80a0394c3'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:26.198457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1203/2406] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:26 2022] DELETE /identity/v3/users/a0c0becb185f49ff86242fe80a0394c3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:26.202919 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8d3e8-632d-45d9-a17a-5745c9f1765f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:26.203211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8d3e8-632d-45d9-a17a-5745c9f1765f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:26.203489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8d3e8-632d-45d9-a17a-5745c9f1765f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:26.221570 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7ed8d3e8-632d-45d9-a17a-5745c9f1765f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:26.222809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7ed8d3e8-632d-45d9-a17a-5745c9f1765f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:26.253961 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:26.291933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1204/2407] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:26.301401 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232105e0-9ffd-473a-bb7e-458c0fcf736b admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:26.303030 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-232105e0-9ffd-473a-bb7e-458c0fcf736b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:26.378032 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232105e0-9ffd-473a-bb7e-458c0fcf736b admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:26.383029 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d77e6-3aa5-4a5d-b012-1440e36a35ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:26.383835 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d77e6-3aa5-4a5d-b012-1440e36a35ee service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:26.384057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d77e6-3aa5-4a5d-b012-1440e36a35ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:26.384405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d77e6-3aa5-4a5d-b012-1440e36a35ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:26.396812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d77e6-3aa5-4a5d-b012-1440e36a35ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:26.398291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d77e6-3aa5-4a5d-b012-1440e36a35ee service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:26.409756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1204/2408] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:26.817386 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8d3e8-632d-45d9-a17a-5745c9f1765f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:26.818975 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7ed8d3e8-632d-45d9-a17a-5745c9f1765f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:26.833025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8d3e8-632d-45d9-a17a-5745c9f1765f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:26.838338 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4def26-759d-4bd1-a38a-2f412b373af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:26.839474 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4def26-759d-4bd1-a38a-2f412b373af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:26.839776 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4def26-759d-4bd1-a38a-2f412b373af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:26.840046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4def26-759d-4bd1-a38a-2f412b373af3 admin admin] PATH_INFO: `/v3/projects/b5a8676f5a1349b08a7741bb53838d16` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:26.841353 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4def26-759d-4bd1-a38a-2f412b373af3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5a8676f5a1349b08a7741bb53838d16)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:26.843262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4def26-759d-4bd1-a38a-2f412b373af3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:26.907272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8c4def26-759d-4bd1-a38a-2f412b373af3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5a8676f5a1349b08a7741bb53838d16', 'request_id': 'req-8c4def26-759d-4bd1-a38a-2f412b373af3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:26.915152 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8c4def26-759d-4bd1-a38a-2f412b373af3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5a8676f5a1349b08a7741bb53838d16', 'request_id': 'req-8c4def26-759d-4bd1-a38a-2f412b373af3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:26.928522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8c4def26-759d-4bd1-a38a-2f412b373af3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5a8676f5a1349b08a7741bb53838d16', 'request_id': 'req-8c4def26-759d-4bd1-a38a-2f412b373af3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:26.952020 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8c4def26-759d-4bd1-a38a-2f412b373af3 admin admin] The token cache is being invalidate because project b5a8676f5a1349b08a7741bb53838d16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:26.952369 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8c4def26-759d-4bd1-a38a-2f412b373af3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:26.953142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8c4def26-759d-4bd1-a38a-2f412b373af3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:26.958486 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1205/2409] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:26 2022] DELETE /identity/v3/projects/b5a8676f5a1349b08a7741bb53838d16 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:26.960398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef9861-473a-43da-98e5-cbf0807413e5 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:26.960845 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef9861-473a-43da-98e5-cbf0807413e5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:26.960941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef9861-473a-43da-98e5-cbf0807413e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:26.979766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-10ef9861-473a-43da-98e5-cbf0807413e5 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:26.981736 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-10ef9861-473a-43da-98e5-cbf0807413e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:27.018071 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:27.068002 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1205/2410] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:27.075748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448071c-4af6-4281-8d5e-ce983d0c686e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:27.076033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448071c-4af6-4281-8d5e-ce983d0c686e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:27.076332 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448071c-4af6-4281-8d5e-ce983d0c686e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:27.105988 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-6448071c-4af6-4281-8d5e-ce983d0c686e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:27.107558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6448071c-4af6-4281-8d5e-ce983d0c686e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:27.118956 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1206/2411] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:27.123720 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef9861-473a-43da-98e5-cbf0807413e5 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:27.139244 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef9861-473a-43da-98e5-cbf0807413e5 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:27.148080 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8800c2a-3d0d-499e-910b-0270d30158ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:27.149382 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8800c2a-3d0d-499e-910b-0270d30158ce admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:27.149679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8800c2a-3d0d-499e-910b-0270d30158ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:27.149987 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8800c2a-3d0d-499e-910b-0270d30158ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:27.150841 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8800c2a-3d0d-499e-910b-0270d30158ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:29:27.151998 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8800c2a-3d0d-499e-910b-0270d30158ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:27.154094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8800c2a-3d0d-499e-910b-0270d30158ce admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:27.172338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1206/2412] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:29:27.177094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448071c-4af6-4281-8d5e-ce983d0c686e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:27.190498 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448071c-4af6-4281-8d5e-ce983d0c686e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:27.195473 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cc8a7-2d84-4e38-86f0-462c3c48afe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:27.196758 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cc8a7-2d84-4e38-86f0-462c3c48afe1 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:27.197115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cc8a7-2d84-4e38-86f0-462c3c48afe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:27.197420 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cc8a7-2d84-4e38-86f0-462c3c48afe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:27.198159 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869cc8a7-2d84-4e38-86f0-462c3c48afe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:27.201071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869cc8a7-2d84-4e38-86f0-462c3c48afe1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:27.245007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1207/2413] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:27 2022] POST /identity/v3/projects => generated 365 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:27.249846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8800c2a-3d0d-499e-910b-0270d30158ce admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:27.260820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8800c2a-3d0d-499e-910b-0270d30158ce admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:27.328807 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48128a65-a28f-43ef-a8bc-c516683dfad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:27.329620 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48128a65-a28f-43ef-a8bc-c516683dfad3 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:27.329946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48128a65-a28f-43ef-a8bc-c516683dfad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:27.330337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48128a65-a28f-43ef-a8bc-c516683dfad3 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:27.331138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48128a65-a28f-43ef-a8bc-c516683dfad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:27.337131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48128a65-a28f-43ef-a8bc-c516683dfad3 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:27.359996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1207/2414] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:27 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:27.361460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cc8a7-2d84-4e38-86f0-462c3c48afe1 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:27.374279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cc8a7-2d84-4e38-86f0-462c3c48afe1 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:27.384639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb59c3-e107-496e-a40c-255d3a0a637a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:27.386941 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb59c3-e107-496e-a40c-255d3a0a637a admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:27.387251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb59c3-e107-496e-a40c-255d3a0a637a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:27.387498 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb59c3-e107-496e-a40c-255d3a0a637a admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:27.389871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb59c3-e107-496e-a40c-255d3a0a637a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:27.392057 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb59c3-e107-496e-a40c-255d3a0a637a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:27.406605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1208/2415] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:27.407801 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48128a65-a28f-43ef-a8bc-c516683dfad3 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:27.418926 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48128a65-a28f-43ef-a8bc-c516683dfad3 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:27.424029 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38123929-a03c-4a53-85d0-c94f9e66d400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:27.424805 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38123929-a03c-4a53-85d0-c94f9e66d400 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:27.425155 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38123929-a03c-4a53-85d0-c94f9e66d400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:27.425496 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38123929-a03c-4a53-85d0-c94f9e66d400 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:27.426289 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38123929-a03c-4a53-85d0-c94f9e66d400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:27.428020 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38123929-a03c-4a53-85d0-c94f9e66d400 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:27.443912 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1208/2416] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:27.445219 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb59c3-e107-496e-a40c-255d3a0a637a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:27.456202 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb59c3-e107-496e-a40c-255d3a0a637a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:27.461941 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b46960-8d30-490e-8436-1688a29e7c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:27.462908 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b46960-8d30-490e-8436-1688a29e7c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:27.463211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b46960-8d30-490e-8436-1688a29e7c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:27.463638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b46960-8d30-490e-8436-1688a29e7c8f admin admin] PATH_INFO: `/v3/projects/098b6cf9bc9a4ba386464af47a06f6b2/users/96e561556c424d5182507fdc693c0fa6/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:27.480840 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b46960-8d30-490e-8436-1688a29e7c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=098b6cf9bc9a4ba386464af47a06f6b2, user_id=96e561556c424d5182507fdc693c0fa6, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:27.483554 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b46960-8d30-490e-8436-1688a29e7c8f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:27.503701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1209/2417] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:27 2022] PUT /identity/v3/projects/098b6cf9bc9a4ba386464af47a06f6b2/users/96e561556c424d5182507fdc693c0fa6/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:27.515867 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57413207-48dc-4019-ac55-e8c8d8c0ad02 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:27.516255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57413207-48dc-4019-ac55-e8c8d8c0ad02 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:27.516721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57413207-48dc-4019-ac55-e8c8d8c0ad02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:27.557441 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-57413207-48dc-4019-ac55-e8c8d8c0ad02 None None] MFA Rules not processed for user `96e561556c424d5182507fdc693c0fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:27.559071 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-57413207-48dc-4019-ac55-e8c8d8c0ad02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:27.635359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1209/2418] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:29:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:29:27.642498 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b46960-8d30-490e-8436-1688a29e7c8f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:27.644305 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b7b46960-8d30-490e-8436-1688a29e7c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:27.712651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b46960-8d30-490e-8436-1688a29e7c8f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:27.716571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375041a6-5904-40db-a092-3c9fd13229f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:27.717233 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375041a6-5904-40db-a092-3c9fd13229f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:27.717519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375041a6-5904-40db-a092-3c9fd13229f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:27.717795 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375041a6-5904-40db-a092-3c9fd13229f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:27.725786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:27.730742 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375041a6-5904-40db-a092-3c9fd13229f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:27.732641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375041a6-5904-40db-a092-3c9fd13229f4 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:27.757904 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1210/2419] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:27 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:29.221210 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57413207-48dc-4019-ac55-e8c8d8c0ad02 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:29.222685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-57413207-48dc-4019-ac55-e8c8d8c0ad02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:29.255433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:29.323389 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57413207-48dc-4019-ac55-e8c8d8c0ad02 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:29.329746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df1043-0002-4eb3-9443-1a5a8bc9b5d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:29.330525 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df1043-0002-4eb3-9443-1a5a8bc9b5d9 service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:29.330904 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df1043-0002-4eb3-9443-1a5a8bc9b5d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:29.331271 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df1043-0002-4eb3-9443-1a5a8bc9b5d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:29.333181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-f4df1043-0002-4eb3-9443-1a5a8bc9b5d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:29.427757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df1043-0002-4eb3-9443-1a5a8bc9b5d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:29.429932 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df1043-0002-4eb3-9443-1a5a8bc9b5d9 service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:29.441852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1210/2420] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:35.779399 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375041a6-5904-40db-a092-3c9fd13229f4 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:35.781276 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-375041a6-5904-40db-a092-3c9fd13229f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:35.844170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375041a6-5904-40db-a092-3c9fd13229f4 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:35.848248 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c97b3-7f0a-4831-ac50-dcbc9933a262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:35.848938 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c97b3-7f0a-4831-ac50-dcbc9933a262 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:35.849206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c97b3-7f0a-4831-ac50-dcbc9933a262 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:35.849497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c97b3-7f0a-4831-ac50-dcbc9933a262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:35.851341 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-329c97b3-7f0a-4831-ac50-dcbc9933a262 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:35.862496 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c97b3-7f0a-4831-ac50-dcbc9933a262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:35.863921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c97b3-7f0a-4831-ac50-dcbc9933a262 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:35.872649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1211/2421] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:36.677496 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df1043-0002-4eb3-9443-1a5a8bc9b5d9 service nova] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:36.744775 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df1043-0002-4eb3-9443-1a5a8bc9b5d9 service nova] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:36.749293 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb175618-09f6-4166-a53e-d3e9cc6862ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:36.750165 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb175618-09f6-4166-a53e-d3e9cc6862ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:36.750456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb175618-09f6-4166-a53e-d3e9cc6862ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:36.750764 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb175618-09f6-4166-a53e-d3e9cc6862ee admin admin] PATH_INFO: `/v3/users/96e561556c424d5182507fdc693c0fa6` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:36.751937 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb175618-09f6-4166-a53e-d3e9cc6862ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=96e561556c424d5182507fdc693c0fa6)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:36.753895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb175618-09f6-4166-a53e-d3e9cc6862ee admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:36.798847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-eb175618-09f6-4166-a53e-d3e9cc6862ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96e561556c424d5182507fdc693c0fa6'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:36.805837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-eb175618-09f6-4166-a53e-d3e9cc6862ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96e561556c424d5182507fdc693c0fa6'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:36.813653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:36.819939 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-eb175618-09f6-4166-a53e-d3e9cc6862ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96e561556c424d5182507fdc693c0fa6'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:36.828478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-eb175618-09f6-4166-a53e-d3e9cc6862ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96e561556c424d5182507fdc693c0fa6'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:36.829144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-eb175618-09f6-4166-a53e-d3e9cc6862ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96e561556c424d5182507fdc693c0fa6'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:36.835073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1211/2422] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:36 2022] DELETE /identity/v3/users/96e561556c424d5182507fdc693c0fa6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:36.839748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4445d8c-48c0-4413-8153-066c3fa05706 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:36.840205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4445d8c-48c0-4413-8153-066c3fa05706 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:36.840333 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4445d8c-48c0-4413-8153-066c3fa05706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:36.857269 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a4445d8c-48c0-4413-8153-066c3fa05706 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:36.858471 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a4445d8c-48c0-4413-8153-066c3fa05706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:36.920451 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1212/2423] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:36.930120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb175618-09f6-4166-a53e-d3e9cc6862ee admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:36.931507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-eb175618-09f6-4166-a53e-d3e9cc6862ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:36.987724 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb175618-09f6-4166-a53e-d3e9cc6862ee admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:36.991319 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2a3b6-dae7-4fb5-b991-e9123232f824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:36.991955 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2a3b6-dae7-4fb5-b991-e9123232f824 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:36.992235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2a3b6-dae7-4fb5-b991-e9123232f824 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:36.992537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2a3b6-dae7-4fb5-b991-e9123232f824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.001916 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2a3b6-dae7-4fb5-b991-e9123232f824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:37.003316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2a3b6-dae7-4fb5-b991-e9123232f824 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:37.013394 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1212/2424] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:37.273187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4445d8c-48c0-4413-8153-066c3fa05706 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:37.274640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a4445d8c-48c0-4413-8153-066c3fa05706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:37.286553 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4445d8c-48c0-4413-8153-066c3fa05706 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:37.292075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d00f1d-6e19-483a-89bd-ca13b8214cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:37.293857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d00f1d-6e19-483a-89bd-ca13b8214cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.294135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d00f1d-6e19-483a-89bd-ca13b8214cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.294558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d00f1d-6e19-483a-89bd-ca13b8214cde admin admin] PATH_INFO: `/v3/projects/098b6cf9bc9a4ba386464af47a06f6b2` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.295712 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d00f1d-6e19-483a-89bd-ca13b8214cde admin admin] RBAC: Authorizing `identity:delete_project(project_id=098b6cf9bc9a4ba386464af47a06f6b2)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:37.298490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d00f1d-6e19-483a-89bd-ca13b8214cde admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:37.348539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-32d00f1d-6e19-483a-89bd-ca13b8214cde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '098b6cf9bc9a4ba386464af47a06f6b2', 'request_id': 'req-32d00f1d-6e19-483a-89bd-ca13b8214cde'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:37.355476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-32d00f1d-6e19-483a-89bd-ca13b8214cde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '098b6cf9bc9a4ba386464af47a06f6b2', 'request_id': 'req-32d00f1d-6e19-483a-89bd-ca13b8214cde'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:37.363107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-32d00f1d-6e19-483a-89bd-ca13b8214cde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '098b6cf9bc9a4ba386464af47a06f6b2', 'request_id': 'req-32d00f1d-6e19-483a-89bd-ca13b8214cde'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:37.373970 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-32d00f1d-6e19-483a-89bd-ca13b8214cde admin admin] The token cache is being invalidate because project 098b6cf9bc9a4ba386464af47a06f6b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:37.374258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-32d00f1d-6e19-483a-89bd-ca13b8214cde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:37.374884 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-32d00f1d-6e19-483a-89bd-ca13b8214cde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:37.376460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1213/2425] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:37 2022] DELETE /identity/v3/projects/098b6cf9bc9a4ba386464af47a06f6b2 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:37.381456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57cb8d-5358-4ba2-8c21-b2c5445243d2 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.382814 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57cb8d-5358-4ba2-8c21-b2c5445243d2 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.383174 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57cb8d-5358-4ba2-8c21-b2c5445243d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.401272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-fc57cb8d-5358-4ba2-8c21-b2c5445243d2 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:37.402391 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-fc57cb8d-5358-4ba2-8c21-b2c5445243d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:37.460893 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1213/2426] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:37.468768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b0a36-9ee5-4994-82cb-c0fa26b50839 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.469060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b0a36-9ee5-4994-82cb-c0fa26b50839 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.469346 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b0a36-9ee5-4994-82cb-c0fa26b50839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.488092 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b83b0a36-9ee5-4994-82cb-c0fa26b50839 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:37.489327 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b83b0a36-9ee5-4994-82cb-c0fa26b50839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:37.497763 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1214/2427] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:37.501831 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57cb8d-5358-4ba2-8c21-b2c5445243d2 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:37.511676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57cb8d-5358-4ba2-8c21-b2c5445243d2 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:37.516188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40892547-cc49-47d0-8051-2a0b4f034e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:37.516871 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40892547-cc49-47d0-8051-2a0b4f034e60 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.517161 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40892547-cc49-47d0-8051-2a0b4f034e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.517446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40892547-cc49-47d0-8051-2a0b4f034e60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.518181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40892547-cc49-47d0-8051-2a0b4f034e60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:29:37.518498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40892547-cc49-47d0-8051-2a0b4f034e60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:37.520107 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40892547-cc49-47d0-8051-2a0b4f034e60 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:37.532158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1214/2428] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:29:37.536358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b0a36-9ee5-4994-82cb-c0fa26b50839 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:37.546134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b0a36-9ee5-4994-82cb-c0fa26b50839 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:37.550809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f7075-4a72-4a7f-b1ad-3cbaf2bb4522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:37.551462 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f7075-4a72-4a7f-b1ad-3cbaf2bb4522 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.551745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f7075-4a72-4a7f-b1ad-3cbaf2bb4522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.552025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f7075-4a72-4a7f-b1ad-3cbaf2bb4522 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.552772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f7075-4a72-4a7f-b1ad-3cbaf2bb4522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:37.554667 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f7075-4a72-4a7f-b1ad-3cbaf2bb4522 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:37.564764 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:37.581063 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1215/2429] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:37 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:37.585269 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40892547-cc49-47d0-8051-2a0b4f034e60 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:37.595611 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40892547-cc49-47d0-8051-2a0b4f034e60 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:37.653313 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741250aa-3a6f-4969-8a81-801f0c60dd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:37.653910 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741250aa-3a6f-4969-8a81-801f0c60dd36 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.654198 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741250aa-3a6f-4969-8a81-801f0c60dd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.654478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741250aa-3a6f-4969-8a81-801f0c60dd36 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.655189 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741250aa-3a6f-4969-8a81-801f0c60dd36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:37.657139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741250aa-3a6f-4969-8a81-801f0c60dd36 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:37.673802 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:37.680275 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1215/2430] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:37 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:37.683915 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f7075-4a72-4a7f-b1ad-3cbaf2bb4522 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:37.694585 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f7075-4a72-4a7f-b1ad-3cbaf2bb4522 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:37.700158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74e486-043b-45c5-b484-0824cd0eacdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:37.700865 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74e486-043b-45c5-b484-0824cd0eacdf admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.701157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74e486-043b-45c5-b484-0824cd0eacdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.701438 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74e486-043b-45c5-b484-0824cd0eacdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.702206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74e486-043b-45c5-b484-0824cd0eacdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:37.703794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74e486-043b-45c5-b484-0824cd0eacdf admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:37.715580 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1216/2431] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:37.719934 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741250aa-3a6f-4969-8a81-801f0c60dd36 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:37.731621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741250aa-3a6f-4969-8a81-801f0c60dd36 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:37.736754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7113030-b866-4135-b04b-2ce20e5f53fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:37.737419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7113030-b866-4135-b04b-2ce20e5f53fb admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.737706 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7113030-b866-4135-b04b-2ce20e5f53fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.737990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7113030-b866-4135-b04b-2ce20e5f53fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.738730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7113030-b866-4135-b04b-2ce20e5f53fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:37.740314 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7113030-b866-4135-b04b-2ce20e5f53fb admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:37.751046 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1216/2432] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:29:37.755065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74e486-043b-45c5-b484-0824cd0eacdf admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:37.765094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74e486-043b-45c5-b484-0824cd0eacdf admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:37.769821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbd181-5b14-4a89-9437-6aee642d6204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:37.770694 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbd181-5b14-4a89-9437-6aee642d6204 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.770972 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbd181-5b14-4a89-9437-6aee642d6204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.771244 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbd181-5b14-4a89-9437-6aee642d6204 admin admin] PATH_INFO: `/v3/projects/8b2080ce58d04c7aaf9a3d0249f5c031/users/677acae3105b439082a27ab4417862dd/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.786607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbd181-5b14-4a89-9437-6aee642d6204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2080ce58d04c7aaf9a3d0249f5c031, user_id=677acae3105b439082a27ab4417862dd, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:37.789534 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbd181-5b14-4a89-9437-6aee642d6204 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:37.803936 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1217/2433] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:37 2022] PUT /identity/v3/projects/8b2080ce58d04c7aaf9a3d0249f5c031/users/677acae3105b439082a27ab4417862dd/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:37.814371 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47587251-70b2-491e-b687-2f8fb9547db8 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.815030 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47587251-70b2-491e-b687-2f8fb9547db8 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.815387 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47587251-70b2-491e-b687-2f8fb9547db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.844254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-47587251-70b2-491e-b687-2f8fb9547db8 None None] MFA Rules not processed for user `677acae3105b439082a27ab4417862dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:37.845686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-47587251-70b2-491e-b687-2f8fb9547db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:37.912737 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1217/2434] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:29:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:37.917126 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbd181-5b14-4a89-9437-6aee642d6204 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:37.978276 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbd181-5b14-4a89-9437-6aee642d6204 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:37.983030 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fde43a-1863-435b-9ebc-266951d04cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:37.983692 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fde43a-1863-435b-9ebc-266951d04cda admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.983980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fde43a-1863-435b-9ebc-266951d04cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.984248 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fde43a-1863-435b-9ebc-266951d04cda admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.984961 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fde43a-1863-435b-9ebc-266951d04cda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:37.986841 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fde43a-1863-435b-9ebc-266951d04cda admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:38.010771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1218/2435] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:37 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:38.015376 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47587251-70b2-491e-b687-2f8fb9547db8 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:38.026416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47587251-70b2-491e-b687-2f8fb9547db8 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:38.082115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb727b3d-57f2-40bb-930b-dd6ea53245fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:38.082780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb727b3d-57f2-40bb-930b-dd6ea53245fe admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:38.083060 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb727b3d-57f2-40bb-930b-dd6ea53245fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:38.083330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb727b3d-57f2-40bb-930b-dd6ea53245fe admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:38.084022 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb727b3d-57f2-40bb-930b-dd6ea53245fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:38.085945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb727b3d-57f2-40bb-930b-dd6ea53245fe admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:38.105324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1218/2436] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:38 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:38.109512 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fde43a-1863-435b-9ebc-266951d04cda admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:38.119658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fde43a-1863-435b-9ebc-266951d04cda admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:38.124452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30c7de-0288-4328-9ff9-e27682deb0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:38.125094 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30c7de-0288-4328-9ff9-e27682deb0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:38.125374 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30c7de-0288-4328-9ff9-e27682deb0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:38.125644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30c7de-0288-4328-9ff9-e27682deb0d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:38.126386 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30c7de-0288-4328-9ff9-e27682deb0d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:38.127955 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30c7de-0288-4328-9ff9-e27682deb0d4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:38.138730 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1219/2437] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:38.142690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb727b3d-57f2-40bb-930b-dd6ea53245fe admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:38.153431 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb727b3d-57f2-40bb-930b-dd6ea53245fe admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:38.158092 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad15c1-cc00-4a92-9286-0ef9c1097fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:38.158966 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad15c1-cc00-4a92-9286-0ef9c1097fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:38.159249 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad15c1-cc00-4a92-9286-0ef9c1097fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:38.159525 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad15c1-cc00-4a92-9286-0ef9c1097fd0 admin admin] PATH_INFO: `/v3/projects/801d128ca20a4e6f80410bb1faf6e965/users/58f40de9c9164c2cac4dd82090b39f06/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:38.173955 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad15c1-cc00-4a92-9286-0ef9c1097fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=801d128ca20a4e6f80410bb1faf6e965, user_id=58f40de9c9164c2cac4dd82090b39f06, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:38.176573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad15c1-cc00-4a92-9286-0ef9c1097fd0 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:38.190174 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1219/2438] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:38 2022] PUT /identity/v3/projects/801d128ca20a4e6f80410bb1faf6e965/users/58f40de9c9164c2cac4dd82090b39f06/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:38.194152 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30c7de-0288-4328-9ff9-e27682deb0d4 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:38.204552 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30c7de-0288-4328-9ff9-e27682deb0d4 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:38.259357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a2561-6369-4021-88df-fadbf88d98f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:38.260043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a2561-6369-4021-88df-fadbf88d98f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:38.260437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a2561-6369-4021-88df-fadbf88d98f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:38.260893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a2561-6369-4021-88df-fadbf88d98f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:38.262096 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a2561-6369-4021-88df-fadbf88d98f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:38.263791 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a2561-6369-4021-88df-fadbf88d98f5 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:38.273573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1220/2439] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:38.278425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad15c1-cc00-4a92-9286-0ef9c1097fd0 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:38.288614 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad15c1-cc00-4a92-9286-0ef9c1097fd0 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:38.293464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54414a-6986-4a36-973f-f702611ce389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:38.294338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54414a-6986-4a36-973f-f702611ce389 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:38.294621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54414a-6986-4a36-973f-f702611ce389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:38.294898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54414a-6986-4a36-973f-f702611ce389 admin admin] PATH_INFO: `/v3/projects/801d128ca20a4e6f80410bb1faf6e965/users/58f40de9c9164c2cac4dd82090b39f06/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:38.296953 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54414a-6986-4a36-973f-f702611ce389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=801d128ca20a4e6f80410bb1faf6e965, user_id=58f40de9c9164c2cac4dd82090b39f06, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:38.299606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54414a-6986-4a36-973f-f702611ce389 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:38.310171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1220/2440] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:38 2022] PUT /identity/v3/projects/801d128ca20a4e6f80410bb1faf6e965/users/58f40de9c9164c2cac4dd82090b39f06/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:38.321649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd8dc0-f13c-448a-a4aa-76293fb38d92 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:38.321940 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd8dc0-f13c-448a-a4aa-76293fb38d92 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:38.322222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd8dc0-f13c-448a-a4aa-76293fb38d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:38.350322 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-48bd8dc0-f13c-448a-a4aa-76293fb38d92 None None] MFA Rules not processed for user `58f40de9c9164c2cac4dd82090b39f06`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:38.351550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-48bd8dc0-f13c-448a-a4aa-76293fb38d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:38.364580 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:38.423258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1221/2441] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:29:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:38.435077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54414a-6986-4a36-973f-f702611ce389 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:38.436902 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ac54414a-6986-4a36-973f-f702611ce389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:38.475002 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:38.500785 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54414a-6986-4a36-973f-f702611ce389 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:38.504464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45b8d9-8312-48d5-82f1-d29d5d2ee68f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:38.505113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45b8d9-8312-48d5-82f1-d29d5d2ee68f service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:38.505404 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45b8d9-8312-48d5-82f1-d29d5d2ee68f service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:38.505682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45b8d9-8312-48d5-82f1-d29d5d2ee68f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:38.515629 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45b8d9-8312-48d5-82f1-d29d5d2ee68f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:38.517090 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45b8d9-8312-48d5-82f1-d29d5d2ee68f service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:38.529861 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1221/2442] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:38.597050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd8dc0-f13c-448a-a4aa-76293fb38d92 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:38.606914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd8dc0-f13c-448a-a4aa-76293fb38d92 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:38.610607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb200d-c497-4137-be63-a26e58e957b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:38.611239 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb200d-c497-4137-be63-a26e58e957b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:38.611522 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb200d-c497-4137-be63-a26e58e957b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:38.611804 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb200d-c497-4137-be63-a26e58e957b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:38.671218 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb200d-c497-4137-be63-a26e58e957b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:38.672693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb200d-c497-4137-be63-a26e58e957b6 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:38.681883 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1222/2443] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:39.851192 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45b8d9-8312-48d5-82f1-d29d5d2ee68f service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:39.852780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-af45b8d9-8312-48d5-82f1-d29d5d2ee68f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:39.936727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45b8d9-8312-48d5-82f1-d29d5d2ee68f service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:39.946963 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0080f4-94d3-4128-aca4-61f6245c5b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:39.947631 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0080f4-94d3-4128-aca4-61f6245c5b9a service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:39.948061 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0080f4-94d3-4128-aca4-61f6245c5b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:39.948377 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0080f4-94d3-4128-aca4-61f6245c5b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:39.951846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5c0080f4-94d3-4128-aca4-61f6245c5b9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:39.963964 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0080f4-94d3-4128-aca4-61f6245c5b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:39.966231 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0080f4-94d3-4128-aca4-61f6245c5b9a service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:39.976730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1222/2444] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:40.095256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16563982-09fb-4423-9f7b-934080454490 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:40.095256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16563982-09fb-4423-9f7b-934080454490 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:40.095256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16563982-09fb-4423-9f7b-934080454490 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:40.096039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1223/2445] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jul 20 19:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:29:40.102786 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5bb85-18a5-4996-b013-5cfc728ac580 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:40.103168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5bb85-18a5-4996-b013-5cfc728ac580 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:40.103457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5bb85-18a5-4996-b013-5cfc728ac580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:40.149502 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4ff5bb85-18a5-4996-b013-5cfc728ac580 None None] MFA Rules not processed for user `267b178236e046e4bc694b2da6f86eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:40.150984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4ff5bb85-18a5-4996-b013-5cfc728ac580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:40.283394 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1223/2446] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jul 20 19:29:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:40.302354 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16563982-09fb-4423-9f7b-934080454490 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:40.304981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-16563982-09fb-4423-9f7b-934080454490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:40.383650 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16563982-09fb-4423-9f7b-934080454490 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:40.388949 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7452b-90a3-4982-a5fa-56bdce1daabf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3e28557896e41c893f0de2590bc96d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:40.389733 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7452b-90a3-4982-a5fa-56bdce1daabf service glance] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:40.390029 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7452b-90a3-4982-a5fa-56bdce1daabf service glance] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:40.390309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7452b-90a3-4982-a5fa-56bdce1daabf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:40.401427 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7452b-90a3-4982-a5fa-56bdce1daabf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:40.404221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7452b-90a3-4982-a5fa-56bdce1daabf service glance] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:40.416928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1224/2447] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:40.487207 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbce5c-95c0-4c33-9aef-2a3fd8ebf6ad None None] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:40.487432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbce5c-95c0-4c33-9aef-2a3fd8ebf6ad None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:40.487711 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbce5c-95c0-4c33-9aef-2a3fd8ebf6ad None None] PATH_INFO: `/` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:40.489110 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1224/2448] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jul 20 19:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:29:40.493953 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49bae2-8633-4692-acde-13496ba63bee None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:40.494313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49bae2-8633-4692-acde-13496ba63bee None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:40.494631 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49bae2-8633-4692-acde-13496ba63bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:40.514673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ce49bae2-8633-4692-acde-13496ba63bee None None] MFA Rules not processed for user `74327a10eaab43e0af02050da1a24f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:40.515910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ce49bae2-8633-4692-acde-13496ba63bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:40.522765 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1225/2449] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jul 20 19:29:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:40.535353 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbce5c-95c0-4c33-9aef-2a3fd8ebf6ad None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:40.537276 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-76cbce5c-95c0-4c33-9aef-2a3fd8ebf6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:40.551685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbce5c-95c0-4c33-9aef-2a3fd8ebf6ad None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:40.555813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b17965-e8e6-4a4a-8d3a-783bbd924174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:40.556759 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b17965-e8e6-4a4a-8d3a-783bbd924174 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:40.557097 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b17965-e8e6-4a4a-8d3a-783bbd924174 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:40.557443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b17965-e8e6-4a4a-8d3a-783bbd924174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:40.565486 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b17965-e8e6-4a4a-8d3a-783bbd924174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:40.566976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b17965-e8e6-4a4a-8d3a-783bbd924174 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:40.575454 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1225/2450] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:40.659685 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094e3c3-f724-4254-aa91-d459681d06b9 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:40.659980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094e3c3-f724-4254-aa91-d459681d06b9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:40.660317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094e3c3-f724-4254-aa91-d459681d06b9 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:40.661705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1226/2451] 10.222.0.43 () {54 vars in 921 bytes} [Wed Jul 20 19:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:29:40.666946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874f898-9a8f-4c7a-916f-1e9c94d7cffd None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:40.667242 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874f898-9a8f-4c7a-916f-1e9c94d7cffd None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:40.667525 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874f898-9a8f-4c7a-916f-1e9c94d7cffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:40.688399 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-2874f898-9a8f-4c7a-916f-1e9c94d7cffd None None] MFA Rules not processed for user `267b178236e046e4bc694b2da6f86eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:40.689911 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-2874f898-9a8f-4c7a-916f-1e9c94d7cffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:40.701169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1226/2452] 10.222.0.43 () {58 vars in 1018 bytes} [Wed Jul 20 19:29:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:40.712209 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094e3c3-f724-4254-aa91-d459681d06b9 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:40.723187 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094e3c3-f724-4254-aa91-d459681d06b9 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:40.727395 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ffca5-b0fb-4a9a-bbb6-43db5a0b4c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:40.728031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ffca5-b0fb-4a9a-bbb6-43db5a0b4c94 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:40.728371 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ffca5-b0fb-4a9a-bbb6-43db5a0b4c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:40.728653 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ffca5-b0fb-4a9a-bbb6-43db5a0b4c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:40.739313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ffca5-b0fb-4a9a-bbb6-43db5a0b4c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:40.740719 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ffca5-b0fb-4a9a-bbb6-43db5a0b4c94 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:40.752117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1227/2453] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:40.812977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874f898-9a8f-4c7a-916f-1e9c94d7cffd None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:40.814434 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-2874f898-9a8f-4c7a-916f-1e9c94d7cffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:40.862524 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:40.882198 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874f898-9a8f-4c7a-916f-1e9c94d7cffd None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:40.887899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907b16b-0db3-4223-a5b3-7693539b0154 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae581d3dda354e07894cb5205d1b2c5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:40.888585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907b16b-0db3-4223-a5b3-7693539b0154 service placement] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:40.888872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907b16b-0db3-4223-a5b3-7693539b0154 service placement] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:40.889150 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907b16b-0db3-4223-a5b3-7693539b0154 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:40.890962 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5907b16b-0db3-4223-a5b3-7693539b0154 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:40.902437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5907b16b-0db3-4223-a5b3-7693539b0154 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:40.904004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5907b16b-0db3-4223-a5b3-7693539b0154 service placement] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:40.913827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1227/2454] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:45.077725 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fda98-da38-41c2-a520-90b47235b58a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:45.078165 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fda98-da38-41c2-a520-90b47235b58a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:45.078271 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fda98-da38-41c2-a520-90b47235b58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:45.098250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-305fda98-da38-41c2-a520-90b47235b58a None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:45.099426 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-305fda98-da38-41c2-a520-90b47235b58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:45.127917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:45.153862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1228/2455] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jul 20 19:29:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:45.165958 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907b16b-0db3-4223-a5b3-7693539b0154 service placement] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:45.167610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5907b16b-0db3-4223-a5b3-7693539b0154 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:45.239343 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907b16b-0db3-4223-a5b3-7693539b0154 service placement] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:45.244297 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b86ea-d8b9-4f50-b2f1-229725b41085 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:45.244957 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b86ea-d8b9-4f50-b2f1-229725b41085 service swift] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:45.245268 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b86ea-d8b9-4f50-b2f1-229725b41085 service swift] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:45.245542 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b86ea-d8b9-4f50-b2f1-229725b41085 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:45.255735 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b86ea-d8b9-4f50-b2f1-229725b41085 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:45.257481 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b86ea-d8b9-4f50-b2f1-229725b41085 service swift] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:45.265667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1228/2456] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:29:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:29:55.146929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=10 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:55.262444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=42 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:59.195754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fda98-da38-41c2-a520-90b47235b58a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:59.197540 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-305fda98-da38-41c2-a520-90b47235b58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:59.217176 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fda98-da38-41c2-a520-90b47235b58a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:59.223198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92b46d-2bba-45ce-8fc0-0d695b0e8e55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:59.223858 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92b46d-2bba-45ce-8fc0-0d695b0e8e55 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:59.224272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92b46d-2bba-45ce-8fc0-0d695b0e8e55 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:59.224596 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92b46d-2bba-45ce-8fc0-0d695b0e8e55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:59.237984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2e92b46d-2bba-45ce-8fc0-0d695b0e8e55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:59.265403 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92b46d-2bba-45ce-8fc0-0d695b0e8e55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:59.267646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92b46d-2bba-45ce-8fc0-0d695b0e8e55 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:59.288516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1229/2457] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:03.651701 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1328267-913f-4b7b-aa9a-fe2cafa69a7e None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:03.652682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1328267-913f-4b7b-aa9a-fe2cafa69a7e None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:03.652682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1328267-913f-4b7b-aa9a-fe2cafa69a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:03.670284 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-e1328267-913f-4b7b-aa9a-fe2cafa69a7e None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:03.671488 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e1328267-913f-4b7b-aa9a-fe2cafa69a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:03.678618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1229/2458] 10.222.0.43 () {58 vars in 967 bytes} [Wed Jul 20 19:30:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:03.691847 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92b46d-2bba-45ce-8fc0-0d695b0e8e55 service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:03.702976 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92b46d-2bba-45ce-8fc0-0d695b0e8e55 service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:03.708199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7937c-db35-42e3-ab6e-6586d96dd76a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:03.708916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7937c-db35-42e3-ab6e-6586d96dd76a service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:03.709203 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7937c-db35-42e3-ab6e-6586d96dd76a service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:03.709484 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7937c-db35-42e3-ab6e-6586d96dd76a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:03.717804 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7937c-db35-42e3-ab6e-6586d96dd76a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:03.719383 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7937c-db35-42e3-ab6e-6586d96dd76a service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:03.727616 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1230/2459] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:30:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:30:08.043260 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1328267-913f-4b7b-aa9a-fe2cafa69a7e None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:08.045234 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e1328267-913f-4b7b-aa9a-fe2cafa69a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:08.068273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1328267-913f-4b7b-aa9a-fe2cafa69a7e None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:08.078430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e656c-502d-4592-a4b4-487ee73ff621 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:08.079355 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e656c-502d-4592-a4b4-487ee73ff621 service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:08.079876 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e656c-502d-4592-a4b4-487ee73ff621 service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:08.080254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e656c-502d-4592-a4b4-487ee73ff621 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:08.082991 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-cc4e656c-502d-4592-a4b4-487ee73ff621 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:08.100164 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e656c-502d-4592-a4b4-487ee73ff621 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:08.102297 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e656c-502d-4592-a4b4-487ee73ff621 service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:08.127882 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1230/2460] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:13.724266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=12 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:13.910364 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7937c-db35-42e3-ab6e-6586d96dd76a service swift] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:13.920365 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7937c-db35-42e3-ab6e-6586d96dd76a service swift] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:13.923858 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f596e08-ee74-40f2-992c-0afdf8abb0fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:13.924635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f596e08-ee74-40f2-992c-0afdf8abb0fa service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:13.924918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f596e08-ee74-40f2-992c-0afdf8abb0fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:13.925191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f596e08-ee74-40f2-992c-0afdf8abb0fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:13.926961 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7f596e08-ee74-40f2-992c-0afdf8abb0fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:13.989945 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f596e08-ee74-40f2-992c-0afdf8abb0fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:13.991412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f596e08-ee74-40f2-992c-0afdf8abb0fa service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:14.001982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1231/2461] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:17.376075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e656c-502d-4592-a4b4-487ee73ff621 service nova] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:17.377758 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-cc4e656c-502d-4592-a4b4-487ee73ff621 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:17.398317 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e656c-502d-4592-a4b4-487ee73ff621 service nova] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:17.406117 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1701f-c4d0-4dfd-a838-7798fcb945f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:17.407163 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1701f-c4d0-4dfd-a838-7798fcb945f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:17.407296 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1701f-c4d0-4dfd-a838-7798fcb945f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:17.407622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1701f-c4d0-4dfd-a838-7798fcb945f4 admin admin] PATH_INFO: `/v3/users/b15f8d66d5104aaeae51e6c1fa3b3596` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:17.409476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1701f-c4d0-4dfd-a838-7798fcb945f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b15f8d66d5104aaeae51e6c1fa3b3596)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:17.411423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1701f-c4d0-4dfd-a838-7798fcb945f4 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:17.459795 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a5a1701f-c4d0-4dfd-a838-7798fcb945f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b15f8d66d5104aaeae51e6c1fa3b3596'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:17.467016 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a5a1701f-c4d0-4dfd-a838-7798fcb945f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b15f8d66d5104aaeae51e6c1fa3b3596'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:17.479360 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a5a1701f-c4d0-4dfd-a838-7798fcb945f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b15f8d66d5104aaeae51e6c1fa3b3596'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:17.486088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a5a1701f-c4d0-4dfd-a838-7798fcb945f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b15f8d66d5104aaeae51e6c1fa3b3596'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:17.486774 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-a5a1701f-c4d0-4dfd-a838-7798fcb945f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b15f8d66d5104aaeae51e6c1fa3b3596'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:17.492704 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1231/2462] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:17 2022] DELETE /identity/v3/users/b15f8d66d5104aaeae51e6c1fa3b3596 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:17.669837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f596e08-ee74-40f2-992c-0afdf8abb0fa service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:17.680637 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f596e08-ee74-40f2-992c-0afdf8abb0fa service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:17.737997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3af132-0813-40a8-acb5-ac4c9675aec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:17.738849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3af132-0813-40a8-acb5-ac4c9675aec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:17.739132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3af132-0813-40a8-acb5-ac4c9675aec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:17.739407 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3af132-0813-40a8-acb5-ac4c9675aec4 admin admin] PATH_INFO: `/v3/projects/654a322fff99416595c66a993aeaee17` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:17.740618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3af132-0813-40a8-acb5-ac4c9675aec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=654a322fff99416595c66a993aeaee17)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:17.742526 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3af132-0813-40a8-acb5-ac4c9675aec4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:17.788973 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2d3af132-0813-40a8-acb5-ac4c9675aec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '654a322fff99416595c66a993aeaee17', 'request_id': 'req-2d3af132-0813-40a8-acb5-ac4c9675aec4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:17.794919 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2d3af132-0813-40a8-acb5-ac4c9675aec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '654a322fff99416595c66a993aeaee17', 'request_id': 'req-2d3af132-0813-40a8-acb5-ac4c9675aec4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:17.802539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2d3af132-0813-40a8-acb5-ac4c9675aec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '654a322fff99416595c66a993aeaee17', 'request_id': 'req-2d3af132-0813-40a8-acb5-ac4c9675aec4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:17.813272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2d3af132-0813-40a8-acb5-ac4c9675aec4 admin admin] The token cache is being invalidate because project 654a322fff99416595c66a993aeaee17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:17.813561 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2d3af132-0813-40a8-acb5-ac4c9675aec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:17.814208 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2d3af132-0813-40a8-acb5-ac4c9675aec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:17.816106 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1232/2463] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:17 2022] DELETE /identity/v3/projects/654a322fff99416595c66a993aeaee17 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:17.821064 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953e43d-d4a9-4085-8231-ba9e9e771e04 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:17.821353 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953e43d-d4a9-4085-8231-ba9e9e771e04 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:17.821627 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953e43d-d4a9-4085-8231-ba9e9e771e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:17.839024 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4953e43d-d4a9-4085-8231-ba9e9e771e04 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:17.840204 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4953e43d-d4a9-4085-8231-ba9e9e771e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:17.897449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1232/2464] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:17.905673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770625d8-4b13-44c1-8042-13b00e52404b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:17.905958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770625d8-4b13-44c1-8042-13b00e52404b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:17.906232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770625d8-4b13-44c1-8042-13b00e52404b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:17.926328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-770625d8-4b13-44c1-8042-13b00e52404b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:17.927541 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-770625d8-4b13-44c1-8042-13b00e52404b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:17.935584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1233/2465] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:17.940153 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953e43d-d4a9-4085-8231-ba9e9e771e04 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:17.949800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953e43d-d4a9-4085-8231-ba9e9e771e04 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:17.955353 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17c8ea-43f5-4511-afb6-d7786f8e5704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:17.956011 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17c8ea-43f5-4511-afb6-d7786f8e5704 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:17.956305 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17c8ea-43f5-4511-afb6-d7786f8e5704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:17.956597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17c8ea-43f5-4511-afb6-d7786f8e5704 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:17.957356 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17c8ea-43f5-4511-afb6-d7786f8e5704 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:30:17.957664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17c8ea-43f5-4511-afb6-d7786f8e5704 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:17.959251 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17c8ea-43f5-4511-afb6-d7786f8e5704 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:17.971383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1233/2466] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:30:17.975654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770625d8-4b13-44c1-8042-13b00e52404b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:17.984828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770625d8-4b13-44c1-8042-13b00e52404b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:17.989828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce3a6f-3348-4926-8845-32279f0f5268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:17.990471 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce3a6f-3348-4926-8845-32279f0f5268 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:17.990756 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce3a6f-3348-4926-8845-32279f0f5268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:17.991031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce3a6f-3348-4926-8845-32279f0f5268 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:17.991718 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce3a6f-3348-4926-8845-32279f0f5268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:17.993625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce3a6f-3348-4926-8845-32279f0f5268 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:18.007196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:18.020412 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1234/2467] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:17 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:18.024634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17c8ea-43f5-4511-afb6-d7786f8e5704 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:18.036393 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17c8ea-43f5-4511-afb6-d7786f8e5704 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:18.074045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:18.094346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecee13b-59c5-4656-9f91-4c78517b1ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:18.095019 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecee13b-59c5-4656-9f91-4c78517b1ddf admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:18.095303 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecee13b-59c5-4656-9f91-4c78517b1ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:18.095579 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecee13b-59c5-4656-9f91-4c78517b1ddf admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:18.096277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecee13b-59c5-4656-9f91-4c78517b1ddf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:18.098207 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecee13b-59c5-4656-9f91-4c78517b1ddf admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:18.121430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1234/2468] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:18 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:30:18.123599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce3a6f-3348-4926-8845-32279f0f5268 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:18.134305 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce3a6f-3348-4926-8845-32279f0f5268 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:18.141618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad268151-0b44-4f7b-9532-ced90af1dcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:18.141618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad268151-0b44-4f7b-9532-ced90af1dcac admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:18.141618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad268151-0b44-4f7b-9532-ced90af1dcac admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:18.141930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad268151-0b44-4f7b-9532-ced90af1dcac admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:18.142588 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad268151-0b44-4f7b-9532-ced90af1dcac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:18.144214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad268151-0b44-4f7b-9532-ced90af1dcac admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:18.155491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1235/2469] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:18.159828 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecee13b-59c5-4656-9f91-4c78517b1ddf admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:18.170025 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecee13b-59c5-4656-9f91-4c78517b1ddf admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:18.175282 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89954201-300a-49ab-a17b-a58eaceb7fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:18.175938 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89954201-300a-49ab-a17b-a58eaceb7fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:18.176220 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89954201-300a-49ab-a17b-a58eaceb7fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:18.179330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89954201-300a-49ab-a17b-a58eaceb7fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:18.179330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89954201-300a-49ab-a17b-a58eaceb7fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:18.179330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89954201-300a-49ab-a17b-a58eaceb7fe2 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:18.194793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1235/2470] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:18.195925 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad268151-0b44-4f7b-9532-ced90af1dcac admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:18.206658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad268151-0b44-4f7b-9532-ced90af1dcac admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:18.211345 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df6a1b-5aae-4e3d-a99d-13d96a549069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:18.212222 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df6a1b-5aae-4e3d-a99d-13d96a549069 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:18.212534 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df6a1b-5aae-4e3d-a99d-13d96a549069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:18.212826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df6a1b-5aae-4e3d-a99d-13d96a549069 admin admin] PATH_INFO: `/v3/projects/bc7e50372ced4f049a19b02fb90670fb/users/1e1c36ee96db4be3a416793b1e72e692/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:18.227397 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df6a1b-5aae-4e3d-a99d-13d96a549069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc7e50372ced4f049a19b02fb90670fb, user_id=1e1c36ee96db4be3a416793b1e72e692, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:18.230076 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df6a1b-5aae-4e3d-a99d-13d96a549069 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:18.243622 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1236/2471] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:18 2022] PUT /identity/v3/projects/bc7e50372ced4f049a19b02fb90670fb/users/1e1c36ee96db4be3a416793b1e72e692/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:18.254007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26916a7b-344c-4bab-9f33-7bc79ce87315 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:18.254303 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26916a7b-344c-4bab-9f33-7bc79ce87315 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:18.254584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26916a7b-344c-4bab-9f33-7bc79ce87315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:18.284886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-26916a7b-344c-4bab-9f33-7bc79ce87315 None None] MFA Rules not processed for user `1e1c36ee96db4be3a416793b1e72e692`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:18.286140 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-26916a7b-344c-4bab-9f33-7bc79ce87315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:18.356317 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1236/2472] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:18.360355 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df6a1b-5aae-4e3d-a99d-13d96a549069 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:18.431015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df6a1b-5aae-4e3d-a99d-13d96a549069 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:18.436501 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab42d5-9ca6-4e7a-8363-b5c3f658a13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:18.437157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab42d5-9ca6-4e7a-8363-b5c3f658a13d admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:18.437437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab42d5-9ca6-4e7a-8363-b5c3f658a13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:18.437709 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab42d5-9ca6-4e7a-8363-b5c3f658a13d admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:18.438549 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab42d5-9ca6-4e7a-8363-b5c3f658a13d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:18.440796 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab42d5-9ca6-4e7a-8363-b5c3f658a13d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:18.469999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1237/2473] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:18 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:18.471138 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26916a7b-344c-4bab-9f33-7bc79ce87315 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:18.482418 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26916a7b-344c-4bab-9f33-7bc79ce87315 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:18.540586 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640becc-5ee3-4fbf-a969-faedea7ede9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:18.541271 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640becc-5ee3-4fbf-a969-faedea7ede9b admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:18.541556 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640becc-5ee3-4fbf-a969-faedea7ede9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:18.541833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640becc-5ee3-4fbf-a969-faedea7ede9b admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:18.542559 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640becc-5ee3-4fbf-a969-faedea7ede9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:18.545057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640becc-5ee3-4fbf-a969-faedea7ede9b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:18.565596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1237/2474] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:18 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:18.569572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab42d5-9ca6-4e7a-8363-b5c3f658a13d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:18.580307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab42d5-9ca6-4e7a-8363-b5c3f658a13d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:18.585235 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572fd5c-c646-4f95-b319-0d1d50a762db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:18.585896 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572fd5c-c646-4f95-b319-0d1d50a762db admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:18.586189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572fd5c-c646-4f95-b319-0d1d50a762db admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:18.586467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572fd5c-c646-4f95-b319-0d1d50a762db admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:18.587217 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572fd5c-c646-4f95-b319-0d1d50a762db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:18.588855 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572fd5c-c646-4f95-b319-0d1d50a762db admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:18.600616 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1238/2475] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:30:18.604908 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640becc-5ee3-4fbf-a969-faedea7ede9b admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:18.617252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640becc-5ee3-4fbf-a969-faedea7ede9b admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:18.622680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a5341-23c5-45e1-881b-96c567ba88be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:18.623610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a5341-23c5-45e1-881b-96c567ba88be admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:18.623922 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a5341-23c5-45e1-881b-96c567ba88be admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:18.624202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a5341-23c5-45e1-881b-96c567ba88be admin admin] PATH_INFO: `/v3/projects/c4a676566cad414d998558f439d88ec2/users/b11c73f23b664d9db1bf6c34384926d0/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:18.640508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a5341-23c5-45e1-881b-96c567ba88be admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4a676566cad414d998558f439d88ec2, user_id=b11c73f23b664d9db1bf6c34384926d0, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:18.643567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a5341-23c5-45e1-881b-96c567ba88be admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:18.656432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1238/2476] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:18 2022] PUT /identity/v3/projects/c4a676566cad414d998558f439d88ec2/users/b11c73f23b664d9db1bf6c34384926d0/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:18.660774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572fd5c-c646-4f95-b319-0d1d50a762db admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:18.671310 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572fd5c-c646-4f95-b319-0d1d50a762db admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:18.729811 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d782a-c469-437e-98a2-8552f5a9a39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:18.730474 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d782a-c469-437e-98a2-8552f5a9a39b admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:18.730752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d782a-c469-437e-98a2-8552f5a9a39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:18.731044 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d782a-c469-437e-98a2-8552f5a9a39b admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:18.731762 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d782a-c469-437e-98a2-8552f5a9a39b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:18.733393 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d782a-c469-437e-98a2-8552f5a9a39b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:18.743133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1239/2477] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:18.747371 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a5341-23c5-45e1-881b-96c567ba88be admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:18.757656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a5341-23c5-45e1-881b-96c567ba88be admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:18.762262 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbf018-09d5-4f67-a0ee-865a5e92a55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:18.763643 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbf018-09d5-4f67-a0ee-865a5e92a55a admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:18.763926 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbf018-09d5-4f67-a0ee-865a5e92a55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:18.764200 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbf018-09d5-4f67-a0ee-865a5e92a55a admin admin] PATH_INFO: `/v3/projects/c4a676566cad414d998558f439d88ec2/users/b11c73f23b664d9db1bf6c34384926d0/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:18.766175 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbf018-09d5-4f67-a0ee-865a5e92a55a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4a676566cad414d998558f439d88ec2, user_id=b11c73f23b664d9db1bf6c34384926d0, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:18.768842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbf018-09d5-4f67-a0ee-865a5e92a55a admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:18.775054 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:18.781057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1239/2478] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:18 2022] PUT /identity/v3/projects/c4a676566cad414d998558f439d88ec2/users/b11c73f23b664d9db1bf6c34384926d0/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:18.792200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d897f9-651a-4922-9236-1663e229b60c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:18.792532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d897f9-651a-4922-9236-1663e229b60c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:18.792814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d897f9-651a-4922-9236-1663e229b60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:18.821246 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-23d897f9-651a-4922-9236-1663e229b60c None None] MFA Rules not processed for user `b11c73f23b664d9db1bf6c34384926d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:18.822465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-23d897f9-651a-4922-9236-1663e229b60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:18.841381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:18.891524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1240/2479] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:18.902666 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbf018-09d5-4f67-a0ee-865a5e92a55a admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:18.904471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-80bbf018-09d5-4f67-a0ee-865a5e92a55a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:18.965690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbf018-09d5-4f67-a0ee-865a5e92a55a admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:18.969564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982e49a-1742-41eb-80e2-77fd17e93f64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:18.972743 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982e49a-1742-41eb-80e2-77fd17e93f64 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:18.973037 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982e49a-1742-41eb-80e2-77fd17e93f64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:18.973322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982e49a-1742-41eb-80e2-77fd17e93f64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:18.983172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982e49a-1742-41eb-80e2-77fd17e93f64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:18.984661 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982e49a-1742-41eb-80e2-77fd17e93f64 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:18.994431 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1240/2480] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:19.065708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d897f9-651a-4922-9236-1663e229b60c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:19.076006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d897f9-651a-4922-9236-1663e229b60c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:19.079631 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a8ee3-d8e0-4fd5-8f1c-acb483c622ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:19.080267 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a8ee3-d8e0-4fd5-8f1c-acb483c622ac service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:19.080595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a8ee3-d8e0-4fd5-8f1c-acb483c622ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:19.080864 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a8ee3-d8e0-4fd5-8f1c-acb483c622ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:19.140254 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a8ee3-d8e0-4fd5-8f1c-acb483c622ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:19.141747 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a8ee3-d8e0-4fd5-8f1c-acb483c622ac service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:19.150902 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1241/2481] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:19.824406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982e49a-1742-41eb-80e2-77fd17e93f64 service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:19.825806 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6982e49a-1742-41eb-80e2-77fd17e93f64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:19.889655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982e49a-1742-41eb-80e2-77fd17e93f64 service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:19.894652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc499367-e98b-4a35-9ece-8766bb7d6cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:19.895544 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc499367-e98b-4a35-9ece-8766bb7d6cfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:19.895826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc499367-e98b-4a35-9ece-8766bb7d6cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:19.896099 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc499367-e98b-4a35-9ece-8766bb7d6cfb admin admin] PATH_INFO: `/v3/users/677acae3105b439082a27ab4417862dd` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:19.897310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc499367-e98b-4a35-9ece-8766bb7d6cfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=677acae3105b439082a27ab4417862dd)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:19.899190 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc499367-e98b-4a35-9ece-8766bb7d6cfb admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:19.950313 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-cc499367-e98b-4a35-9ece-8766bb7d6cfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '677acae3105b439082a27ab4417862dd'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:19.957828 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-cc499367-e98b-4a35-9ece-8766bb7d6cfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '677acae3105b439082a27ab4417862dd'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:19.971054 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-cc499367-e98b-4a35-9ece-8766bb7d6cfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '677acae3105b439082a27ab4417862dd'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:19.978001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-cc499367-e98b-4a35-9ece-8766bb7d6cfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '677acae3105b439082a27ab4417862dd'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:19.978681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-cc499367-e98b-4a35-9ece-8766bb7d6cfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '677acae3105b439082a27ab4417862dd'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:19.984731 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1241/2482] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:19 2022] DELETE /identity/v3/users/677acae3105b439082a27ab4417862dd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:19.989206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0ba4d-7ef0-45a7-bc17-37da56124438 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:19.989677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0ba4d-7ef0-45a7-bc17-37da56124438 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:19.989795 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0ba4d-7ef0-45a7-bc17-37da56124438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:20.006380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-18d0ba4d-7ef0-45a7-bc17-37da56124438 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:20.007560 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-18d0ba4d-7ef0-45a7-bc17-37da56124438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:20.065438 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1242/2483] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:20.075588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc499367-e98b-4a35-9ece-8766bb7d6cfb admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:20.077004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-cc499367-e98b-4a35-9ece-8766bb7d6cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:20.103366 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:20.135444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc499367-e98b-4a35-9ece-8766bb7d6cfb admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:20.139784 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c24d19-48b8-4197-9405-4a010add92ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:20.140465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c24d19-48b8-4197-9405-4a010add92ca service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:20.140754 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c24d19-48b8-4197-9405-4a010add92ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:20.141026 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c24d19-48b8-4197-9405-4a010add92ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:20.150057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c24d19-48b8-4197-9405-4a010add92ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:20.151469 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c24d19-48b8-4197-9405-4a010add92ca service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:20.161469 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1242/2484] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:20.443628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0ba4d-7ef0-45a7-bc17-37da56124438 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:20.446162 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-18d0ba4d-7ef0-45a7-bc17-37da56124438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:20.461721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0ba4d-7ef0-45a7-bc17-37da56124438 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:20.467088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d767c-95c4-4855-bdcb-dce21333c28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:20.467827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d767c-95c4-4855-bdcb-dce21333c28e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:20.468109 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d767c-95c4-4855-bdcb-dce21333c28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:20.468547 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d767c-95c4-4855-bdcb-dce21333c28e admin admin] PATH_INFO: `/v3/projects/8b2080ce58d04c7aaf9a3d0249f5c031` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:20.470314 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d767c-95c4-4855-bdcb-dce21333c28e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b2080ce58d04c7aaf9a3d0249f5c031)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:20.472145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d767c-95c4-4855-bdcb-dce21333c28e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:20.529332 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9d2d767c-95c4-4855-bdcb-dce21333c28e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b2080ce58d04c7aaf9a3d0249f5c031', 'request_id': 'req-9d2d767c-95c4-4855-bdcb-dce21333c28e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:20.530618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:20.538787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9d2d767c-95c4-4855-bdcb-dce21333c28e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b2080ce58d04c7aaf9a3d0249f5c031', 'request_id': 'req-9d2d767c-95c4-4855-bdcb-dce21333c28e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:20.546070 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9d2d767c-95c4-4855-bdcb-dce21333c28e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b2080ce58d04c7aaf9a3d0249f5c031', 'request_id': 'req-9d2d767c-95c4-4855-bdcb-dce21333c28e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:20.558008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9d2d767c-95c4-4855-bdcb-dce21333c28e admin admin] The token cache is being invalidate because project 8b2080ce58d04c7aaf9a3d0249f5c031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:20.558875 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9d2d767c-95c4-4855-bdcb-dce21333c28e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:20.560181 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9d2d767c-95c4-4855-bdcb-dce21333c28e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:20.561881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1243/2485] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:20 2022] DELETE /identity/v3/projects/8b2080ce58d04c7aaf9a3d0249f5c031 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:20.566478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c24d19-48b8-4197-9405-4a010add92ca service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:20.577187 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c24d19-48b8-4197-9405-4a010add92ca service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:20.667143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038a758-73b6-4b79-9067-8a741988a88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:20.668146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038a758-73b6-4b79-9067-8a741988a88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:20.669030 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038a758-73b6-4b79-9067-8a741988a88a admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:20.669405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038a758-73b6-4b79-9067-8a741988a88a admin admin] PATH_INFO: `/v3/users/58f40de9c9164c2cac4dd82090b39f06` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:20.670849 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038a758-73b6-4b79-9067-8a741988a88a admin admin] RBAC: Authorizing `identity:delete_user(user_id=58f40de9c9164c2cac4dd82090b39f06)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:20.672900 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038a758-73b6-4b79-9067-8a741988a88a admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:20.725437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-f038a758-73b6-4b79-9067-8a741988a88a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58f40de9c9164c2cac4dd82090b39f06'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:20.733415 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-f038a758-73b6-4b79-9067-8a741988a88a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58f40de9c9164c2cac4dd82090b39f06'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:20.745141 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-f038a758-73b6-4b79-9067-8a741988a88a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58f40de9c9164c2cac4dd82090b39f06'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:20.752381 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-f038a758-73b6-4b79-9067-8a741988a88a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58f40de9c9164c2cac4dd82090b39f06'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:20.753112 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-f038a758-73b6-4b79-9067-8a741988a88a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58f40de9c9164c2cac4dd82090b39f06'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:20.759194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1243/2486] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:20 2022] DELETE /identity/v3/users/58f40de9c9164c2cac4dd82090b39f06 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:21.022621 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d767c-95c4-4855-bdcb-dce21333c28e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:21.024569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9d2d767c-95c4-4855-bdcb-dce21333c28e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:21.039781 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d767c-95c4-4855-bdcb-dce21333c28e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:21.097318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e22a8-1e25-4580-9a80-0032fe788261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:21.098204 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e22a8-1e25-4580-9a80-0032fe788261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:21.098483 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e22a8-1e25-4580-9a80-0032fe788261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:21.098755 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e22a8-1e25-4580-9a80-0032fe788261 admin admin] PATH_INFO: `/v3/projects/801d128ca20a4e6f80410bb1faf6e965` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:21.099937 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e22a8-1e25-4580-9a80-0032fe788261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=801d128ca20a4e6f80410bb1faf6e965)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:21.101919 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e22a8-1e25-4580-9a80-0032fe788261 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:21.149369 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-176e22a8-1e25-4580-9a80-0032fe788261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '801d128ca20a4e6f80410bb1faf6e965', 'request_id': 'req-176e22a8-1e25-4580-9a80-0032fe788261'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:21.156027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-176e22a8-1e25-4580-9a80-0032fe788261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '801d128ca20a4e6f80410bb1faf6e965', 'request_id': 'req-176e22a8-1e25-4580-9a80-0032fe788261'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:21.164064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-176e22a8-1e25-4580-9a80-0032fe788261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '801d128ca20a4e6f80410bb1faf6e965', 'request_id': 'req-176e22a8-1e25-4580-9a80-0032fe788261'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:21.175671 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-176e22a8-1e25-4580-9a80-0032fe788261 admin admin] The token cache is being invalidate because project 801d128ca20a4e6f80410bb1faf6e965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:21.175959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-176e22a8-1e25-4580-9a80-0032fe788261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:21.176608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-176e22a8-1e25-4580-9a80-0032fe788261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:21.178191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1244/2487] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:21 2022] DELETE /identity/v3/projects/801d128ca20a4e6f80410bb1faf6e965 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:21.183266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a8cc8-3c9d-4d2d-94f8-d94ba55737b6 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:21.183732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a8cc8-3c9d-4d2d-94f8-d94ba55737b6 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:21.183815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a8cc8-3c9d-4d2d-94f8-d94ba55737b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:21.201378 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-1f6a8cc8-3c9d-4d2d-94f8-d94ba55737b6 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:21.202571 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-1f6a8cc8-3c9d-4d2d-94f8-d94ba55737b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:21.242767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:21.262603 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1244/2488] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:21.270771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45f171-22d7-4e32-afd9-eeba73d823fa None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:21.271007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45f171-22d7-4e32-afd9-eeba73d823fa None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:21.271311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45f171-22d7-4e32-afd9-eeba73d823fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:21.289665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5d45f171-22d7-4e32-afd9-eeba73d823fa None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:21.290847 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5d45f171-22d7-4e32-afd9-eeba73d823fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:21.299390 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1245/2489] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:21.303557 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a8cc8-3c9d-4d2d-94f8-d94ba55737b6 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:21.313444 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a8cc8-3c9d-4d2d-94f8-d94ba55737b6 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:21.318056 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde9049-78c3-4b25-a80f-22faf138efad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:21.318699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde9049-78c3-4b25-a80f-22faf138efad admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:21.318981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde9049-78c3-4b25-a80f-22faf138efad admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:21.319255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde9049-78c3-4b25-a80f-22faf138efad admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:21.319972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde9049-78c3-4b25-a80f-22faf138efad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:30:21.320282 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde9049-78c3-4b25-a80f-22faf138efad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:21.321879 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde9049-78c3-4b25-a80f-22faf138efad admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:21.334002 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1245/2490] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:30:21.338240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45f171-22d7-4e32-afd9-eeba73d823fa None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:21.348986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45f171-22d7-4e32-afd9-eeba73d823fa None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:21.355009 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2c212-7ed5-4cfc-ac99-67edde868a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:21.355009 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2c212-7ed5-4cfc-ac99-67edde868a30 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:21.355009 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2c212-7ed5-4cfc-ac99-67edde868a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:21.357186 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2c212-7ed5-4cfc-ac99-67edde868a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:21.357186 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2c212-7ed5-4cfc-ac99-67edde868a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:21.358613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2c212-7ed5-4cfc-ac99-67edde868a30 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:21.385915 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1246/2491] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:21 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:21.390993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde9049-78c3-4b25-a80f-22faf138efad admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:21.400868 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde9049-78c3-4b25-a80f-22faf138efad admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:21.456487 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ce523-07c4-4b36-a9d8-d068af693f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:21.457154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ce523-07c4-4b36-a9d8-d068af693f36 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:21.457435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ce523-07c4-4b36-a9d8-d068af693f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:21.457708 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ce523-07c4-4b36-a9d8-d068af693f36 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:21.458396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ce523-07c4-4b36-a9d8-d068af693f36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:21.460302 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ce523-07c4-4b36-a9d8-d068af693f36 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:21.481449 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1246/2492] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:21 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:30:21.485812 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2c212-7ed5-4cfc-ac99-67edde868a30 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:21.498492 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2c212-7ed5-4cfc-ac99-67edde868a30 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:21.503327 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1ad8d-e712-4743-b3d0-27ad270844f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:21.503985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1ad8d-e712-4743-b3d0-27ad270844f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:21.504268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1ad8d-e712-4743-b3d0-27ad270844f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:21.504584 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1ad8d-e712-4743-b3d0-27ad270844f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:21.505305 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1ad8d-e712-4743-b3d0-27ad270844f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:21.506963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1ad8d-e712-4743-b3d0-27ad270844f0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:21.518130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1247/2493] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:21.522230 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ce523-07c4-4b36-a9d8-d068af693f36 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:21.532659 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ce523-07c4-4b36-a9d8-d068af693f36 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:21.537194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758ef05-374f-44d7-a0a7-6afba062deeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:21.537843 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758ef05-374f-44d7-a0a7-6afba062deeb admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:21.538125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758ef05-374f-44d7-a0a7-6afba062deeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:21.538398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758ef05-374f-44d7-a0a7-6afba062deeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:21.539113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758ef05-374f-44d7-a0a7-6afba062deeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:21.540734 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758ef05-374f-44d7-a0a7-6afba062deeb admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:21.550500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1247/2494] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:21.554544 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1ad8d-e712-4743-b3d0-27ad270844f0 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:21.565103 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1ad8d-e712-4743-b3d0-27ad270844f0 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:21.570014 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690c286-94a7-431e-93eb-c808b8798373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:21.571003 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690c286-94a7-431e-93eb-c808b8798373 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:21.571288 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690c286-94a7-431e-93eb-c808b8798373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:21.571564 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690c286-94a7-431e-93eb-c808b8798373 admin admin] PATH_INFO: `/v3/projects/65fd7f8b90a445a5aa42f77d592bd93e/users/ce8d7ac1d2fd4546847209ed383e7a99/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:21.586758 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690c286-94a7-431e-93eb-c808b8798373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65fd7f8b90a445a5aa42f77d592bd93e, user_id=ce8d7ac1d2fd4546847209ed383e7a99, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:21.589451 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690c286-94a7-431e-93eb-c808b8798373 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:21.603507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1248/2495] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:21 2022] PUT /identity/v3/projects/65fd7f8b90a445a5aa42f77d592bd93e/users/ce8d7ac1d2fd4546847209ed383e7a99/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:21.614796 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6837b-a744-44e0-ac9d-fd9d2f919e95 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:21.615219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6837b-a744-44e0-ac9d-fd9d2f919e95 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:21.615541 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6837b-a744-44e0-ac9d-fd9d2f919e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:21.642437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-bbd6837b-a744-44e0-ac9d-fd9d2f919e95 None None] MFA Rules not processed for user `ce8d7ac1d2fd4546847209ed383e7a99`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:21.643629 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-bbd6837b-a744-44e0-ac9d-fd9d2f919e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:21.707553 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1248/2496] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:21.713381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690c286-94a7-431e-93eb-c808b8798373 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:21.719931 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:21.784671 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690c286-94a7-431e-93eb-c808b8798373 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:21.789651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c058a-2830-46f6-92e6-6c3908f1039f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:21.790302 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c058a-2830-46f6-92e6-6c3908f1039f admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:21.790581 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c058a-2830-46f6-92e6-6c3908f1039f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:21.790855 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c058a-2830-46f6-92e6-6c3908f1039f admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:21.791541 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c058a-2830-46f6-92e6-6c3908f1039f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:21.793476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c058a-2830-46f6-92e6-6c3908f1039f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:21.824530 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1249/2497] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:21 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:21.827706 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6837b-a744-44e0-ac9d-fd9d2f919e95 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:21.839121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6837b-a744-44e0-ac9d-fd9d2f919e95 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:21.878380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:21.901155 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91407b1-0a5a-4d6f-8a81-3acb0c68b030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:21.901872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91407b1-0a5a-4d6f-8a81-3acb0c68b030 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:21.902167 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91407b1-0a5a-4d6f-8a81-3acb0c68b030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:21.902438 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91407b1-0a5a-4d6f-8a81-3acb0c68b030 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:21.903136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91407b1-0a5a-4d6f-8a81-3acb0c68b030 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:21.905179 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91407b1-0a5a-4d6f-8a81-3acb0c68b030 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:21.927796 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1249/2498] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:21 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:21.929010 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c058a-2830-46f6-92e6-6c3908f1039f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:21.942755 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c058a-2830-46f6-92e6-6c3908f1039f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:21.949016 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb04471-4d70-49af-a1a8-3ee31a5adbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:21.949691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb04471-4d70-49af-a1a8-3ee31a5adbab admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:21.950607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb04471-4d70-49af-a1a8-3ee31a5adbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:21.950883 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb04471-4d70-49af-a1a8-3ee31a5adbab admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:21.951617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb04471-4d70-49af-a1a8-3ee31a5adbab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:21.953266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb04471-4d70-49af-a1a8-3ee31a5adbab admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:21.964516 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1250/2499] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:30:21.968775 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91407b1-0a5a-4d6f-8a81-3acb0c68b030 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:21.978813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91407b1-0a5a-4d6f-8a81-3acb0c68b030 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:21.983388 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d698ac1-16bd-4334-a6d8-391e8577e182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:21.984261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d698ac1-16bd-4334-a6d8-391e8577e182 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:21.984575 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d698ac1-16bd-4334-a6d8-391e8577e182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:21.984847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d698ac1-16bd-4334-a6d8-391e8577e182 admin admin] PATH_INFO: `/v3/projects/d151e2d0269c4ae7af4d6b5057bdbee6/users/31b10c72865b4e53a86927237b729cdc/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:21.999864 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d698ac1-16bd-4334-a6d8-391e8577e182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d151e2d0269c4ae7af4d6b5057bdbee6, user_id=31b10c72865b4e53a86927237b729cdc, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:22.002527 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d698ac1-16bd-4334-a6d8-391e8577e182 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:22.016627 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1250/2500] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:21 2022] PUT /identity/v3/projects/d151e2d0269c4ae7af4d6b5057bdbee6/users/31b10c72865b4e53a86927237b729cdc/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:22.020702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb04471-4d70-49af-a1a8-3ee31a5adbab admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:22.030756 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb04471-4d70-49af-a1a8-3ee31a5adbab admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:22.087175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca80e0-9360-4575-8362-138219a38dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:22.087838 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca80e0-9360-4575-8362-138219a38dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:22.088123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca80e0-9360-4575-8362-138219a38dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:22.088442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca80e0-9360-4575-8362-138219a38dc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:22.089164 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca80e0-9360-4575-8362-138219a38dc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:22.090798 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca80e0-9360-4575-8362-138219a38dc6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:22.101505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1251/2501] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:22.105805 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d698ac1-16bd-4334-a6d8-391e8577e182 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:22.115905 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d698ac1-16bd-4334-a6d8-391e8577e182 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:22.120533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453df70b-d713-4e68-93f6-8340cbaf6c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:22.121399 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453df70b-d713-4e68-93f6-8340cbaf6c26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:22.122183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453df70b-d713-4e68-93f6-8340cbaf6c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:22.122457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453df70b-d713-4e68-93f6-8340cbaf6c26 admin admin] PATH_INFO: `/v3/projects/d151e2d0269c4ae7af4d6b5057bdbee6/users/31b10c72865b4e53a86927237b729cdc/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:22.124434 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453df70b-d713-4e68-93f6-8340cbaf6c26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d151e2d0269c4ae7af4d6b5057bdbee6, user_id=31b10c72865b4e53a86927237b729cdc, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:22.127048 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453df70b-d713-4e68-93f6-8340cbaf6c26 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:22.136921 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1251/2502] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:22 2022] PUT /identity/v3/projects/d151e2d0269c4ae7af4d6b5057bdbee6/users/31b10c72865b4e53a86927237b729cdc/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:22.147368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f9b44-d3c6-4569-8b02-656e92963b6a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:22.147641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f9b44-d3c6-4569-8b02-656e92963b6a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:22.147917 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f9b44-d3c6-4569-8b02-656e92963b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:22.174524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-514f9b44-d3c6-4569-8b02-656e92963b6a None None] MFA Rules not processed for user `31b10c72865b4e53a86927237b729cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:22.175733 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-514f9b44-d3c6-4569-8b02-656e92963b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:22.240920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:22.249680 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1252/2503] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:22.257377 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453df70b-d713-4e68-93f6-8340cbaf6c26 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:22.259177 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-453df70b-d713-4e68-93f6-8340cbaf6c26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:22.317156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453df70b-d713-4e68-93f6-8340cbaf6c26 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:22.320740 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fca19-9125-4b2b-843d-fc099fdd8355 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:22.321383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fca19-9125-4b2b-843d-fc099fdd8355 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:22.321665 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fca19-9125-4b2b-843d-fc099fdd8355 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:22.321940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fca19-9125-4b2b-843d-fc099fdd8355 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:22.330911 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fca19-9125-4b2b-843d-fc099fdd8355 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:22.332340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fca19-9125-4b2b-843d-fc099fdd8355 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:22.341911 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1252/2504] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:22.404651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f9b44-d3c6-4569-8b02-656e92963b6a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:22.467049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f9b44-d3c6-4569-8b02-656e92963b6a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:22.471686 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd86b74-dfce-4c78-8618-77230ebf8b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:22.472692 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd86b74-dfce-4c78-8618-77230ebf8b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:22.472973 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd86b74-dfce-4c78-8618-77230ebf8b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:22.473247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd86b74-dfce-4c78-8618-77230ebf8b93 admin admin] PATH_INFO: `/v3/users/ce8d7ac1d2fd4546847209ed383e7a99` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:22.474460 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd86b74-dfce-4c78-8618-77230ebf8b93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce8d7ac1d2fd4546847209ed383e7a99)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:22.476408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd86b74-dfce-4c78-8618-77230ebf8b93 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:22.524030 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cfd86b74-dfce-4c78-8618-77230ebf8b93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce8d7ac1d2fd4546847209ed383e7a99'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:22.532338 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cfd86b74-dfce-4c78-8618-77230ebf8b93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce8d7ac1d2fd4546847209ed383e7a99'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:22.533301 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cfd86b74-dfce-4c78-8618-77230ebf8b93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce8d7ac1d2fd4546847209ed383e7a99'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:22.540480 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cfd86b74-dfce-4c78-8618-77230ebf8b93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce8d7ac1d2fd4546847209ed383e7a99'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:22.552862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cfd86b74-dfce-4c78-8618-77230ebf8b93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce8d7ac1d2fd4546847209ed383e7a99'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:22.558723 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1253/2505] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:22 2022] DELETE /identity/v3/users/ce8d7ac1d2fd4546847209ed383e7a99 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:22.563243 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309cd87-3632-414b-924b-bdd73f82f5c0 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:22.563534 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309cd87-3632-414b-924b-bdd73f82f5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:22.563816 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309cd87-3632-414b-924b-bdd73f82f5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:22.581451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-a309cd87-3632-414b-924b-bdd73f82f5c0 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:22.582650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-a309cd87-3632-414b-924b-bdd73f82f5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:22.647376 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1253/2506] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:22.657563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd86b74-dfce-4c78-8618-77230ebf8b93 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:22.659097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cfd86b74-dfce-4c78-8618-77230ebf8b93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:22.715240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd86b74-dfce-4c78-8618-77230ebf8b93 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:22.719378 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58bcd8-f558-45e1-a04f-09d6ca68833b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:22.720033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58bcd8-f558-45e1-a04f-09d6ca68833b service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:22.720312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58bcd8-f558-45e1-a04f-09d6ca68833b service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:22.720598 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58bcd8-f558-45e1-a04f-09d6ca68833b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:22.729550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58bcd8-f558-45e1-a04f-09d6ca68833b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:22.730994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58bcd8-f558-45e1-a04f-09d6ca68833b service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:22.741235 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1254/2507] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:23.018177 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309cd87-3632-414b-924b-bdd73f82f5c0 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:23.020505 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-a309cd87-3632-414b-924b-bdd73f82f5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:23.037056 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309cd87-3632-414b-924b-bdd73f82f5c0 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:23.042110 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:23.042979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:23.044358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:23.044650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd admin admin] PATH_INFO: `/v3/projects/65fd7f8b90a445a5aa42f77d592bd93e` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:23.046172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=65fd7f8b90a445a5aa42f77d592bd93e)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:23.048427 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:23.084665 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:23.099303 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65fd7f8b90a445a5aa42f77d592bd93e', 'request_id': 'req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:23.106477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65fd7f8b90a445a5aa42f77d592bd93e', 'request_id': 'req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:23.112283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65fd7f8b90a445a5aa42f77d592bd93e', 'request_id': 'req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:23.123073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd admin admin] The token cache is being invalidate because project 65fd7f8b90a445a5aa42f77d592bd93e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:23.123358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:23.123993 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:23.125537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1254/2508] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:23 2022] DELETE /identity/v3/projects/65fd7f8b90a445a5aa42f77d592bd93e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:23.129593 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58bcd8-f558-45e1-a04f-09d6ca68833b service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:23.139619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58bcd8-f558-45e1-a04f-09d6ca68833b service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:23.157794 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:23.200828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b4ac7-8ed6-4fdf-bb8f-ed093722a349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:23.201688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b4ac7-8ed6-4fdf-bb8f-ed093722a349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:23.201963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b4ac7-8ed6-4fdf-bb8f-ed093722a349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:23.202256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b4ac7-8ed6-4fdf-bb8f-ed093722a349 admin admin] PATH_INFO: `/v3/users/31b10c72865b4e53a86927237b729cdc` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:23.203435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b4ac7-8ed6-4fdf-bb8f-ed093722a349 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31b10c72865b4e53a86927237b729cdc)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:23.205466 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b4ac7-8ed6-4fdf-bb8f-ed093722a349 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:23.249658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ee0b4ac7-8ed6-4fdf-bb8f-ed093722a349 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31b10c72865b4e53a86927237b729cdc'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:23.256656 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ee0b4ac7-8ed6-4fdf-bb8f-ed093722a349 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31b10c72865b4e53a86927237b729cdc'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:23.257346 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ee0b4ac7-8ed6-4fdf-bb8f-ed093722a349 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31b10c72865b4e53a86927237b729cdc'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:23.264824 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ee0b4ac7-8ed6-4fdf-bb8f-ed093722a349 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31b10c72865b4e53a86927237b729cdc'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:23.276759 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ee0b4ac7-8ed6-4fdf-bb8f-ed093722a349 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31b10c72865b4e53a86927237b729cdc'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:23.283267 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1255/2509] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:23 2022] DELETE /identity/v3/users/31b10c72865b4e53a86927237b729cdc => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:23.566038 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:23.567593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:23.583485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b0bea-3b29-4197-9076-c05ceaa7e2bd admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:23.645460 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afd8aa-a0f1-4238-a74f-e163b45f1e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:23.646328 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afd8aa-a0f1-4238-a74f-e163b45f1e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:23.646612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afd8aa-a0f1-4238-a74f-e163b45f1e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:23.646892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afd8aa-a0f1-4238-a74f-e163b45f1e06 admin admin] PATH_INFO: `/v3/projects/d151e2d0269c4ae7af4d6b5057bdbee6` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:23.648093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afd8aa-a0f1-4238-a74f-e163b45f1e06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d151e2d0269c4ae7af4d6b5057bdbee6)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:23.650014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afd8aa-a0f1-4238-a74f-e163b45f1e06 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:23.695679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-31afd8aa-a0f1-4238-a74f-e163b45f1e06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd151e2d0269c4ae7af4d6b5057bdbee6', 'request_id': 'req-31afd8aa-a0f1-4238-a74f-e163b45f1e06'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:23.702969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-31afd8aa-a0f1-4238-a74f-e163b45f1e06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd151e2d0269c4ae7af4d6b5057bdbee6', 'request_id': 'req-31afd8aa-a0f1-4238-a74f-e163b45f1e06'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:23.708896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-31afd8aa-a0f1-4238-a74f-e163b45f1e06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd151e2d0269c4ae7af4d6b5057bdbee6', 'request_id': 'req-31afd8aa-a0f1-4238-a74f-e163b45f1e06'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:23.720204 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-31afd8aa-a0f1-4238-a74f-e163b45f1e06 admin admin] The token cache is being invalidate because project d151e2d0269c4ae7af4d6b5057bdbee6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:23.720543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-31afd8aa-a0f1-4238-a74f-e163b45f1e06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:23.721215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-31afd8aa-a0f1-4238-a74f-e163b45f1e06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:23.722763 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1255/2510] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:23 2022] DELETE /identity/v3/projects/d151e2d0269c4ae7af4d6b5057bdbee6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:23.727652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d209c89-fa85-4483-8029-cc9b0402cd4f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:23.727948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d209c89-fa85-4483-8029-cc9b0402cd4f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:23.728233 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d209c89-fa85-4483-8029-cc9b0402cd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:23.748034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1d209c89-fa85-4483-8029-cc9b0402cd4f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:23.749280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1d209c89-fa85-4483-8029-cc9b0402cd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:23.807200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1256/2511] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:23.815876 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712b0aa-1c41-420a-85c2-80233a843446 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:23.816167 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712b0aa-1c41-420a-85c2-80233a843446 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:23.816485 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712b0aa-1c41-420a-85c2-80233a843446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:23.834549 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-d712b0aa-1c41-420a-85c2-80233a843446 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:23.835767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d712b0aa-1c41-420a-85c2-80233a843446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:23.843769 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1256/2512] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:23.847895 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d209c89-fa85-4483-8029-cc9b0402cd4f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:23.858179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d209c89-fa85-4483-8029-cc9b0402cd4f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:23.862907 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c13a0-c6e0-4483-9e8e-a0f9cd77a8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:23.863565 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c13a0-c6e0-4483-9e8e-a0f9cd77a8de admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:23.863853 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c13a0-c6e0-4483-9e8e-a0f9cd77a8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:23.864158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c13a0-c6e0-4483-9e8e-a0f9cd77a8de admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:23.864933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c13a0-c6e0-4483-9e8e-a0f9cd77a8de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:30:23.865247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c13a0-c6e0-4483-9e8e-a0f9cd77a8de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:23.866909 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c13a0-c6e0-4483-9e8e-a0f9cd77a8de admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:23.879186 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1257/2513] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:30:23.883574 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712b0aa-1c41-420a-85c2-80233a843446 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:23.893181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712b0aa-1c41-420a-85c2-80233a843446 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:23.897773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bc221-24d1-4f62-bb32-816309e8e131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:23.898414 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bc221-24d1-4f62-bb32-816309e8e131 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:23.898700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bc221-24d1-4f62-bb32-816309e8e131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:23.898974 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bc221-24d1-4f62-bb32-816309e8e131 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:23.899658 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bc221-24d1-4f62-bb32-816309e8e131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:23.901568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bc221-24d1-4f62-bb32-816309e8e131 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:23.927218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1257/2514] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:23 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:23.931384 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c13a0-c6e0-4483-9e8e-a0f9cd77a8de admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:23.942297 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c13a0-c6e0-4483-9e8e-a0f9cd77a8de admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:23.967750 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:24.000844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe55765-1ae5-4f4b-a48f-04056cff38eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:24.001518 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe55765-1ae5-4f4b-a48f-04056cff38eb admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:24.001801 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe55765-1ae5-4f4b-a48f-04056cff38eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:24.002078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe55765-1ae5-4f4b-a48f-04056cff38eb admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:24.002771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe55765-1ae5-4f4b-a48f-04056cff38eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:24.004726 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe55765-1ae5-4f4b-a48f-04056cff38eb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:24.029720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1258/2515] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:23 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:24.031396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bc221-24d1-4f62-bb32-816309e8e131 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:24.041735 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bc221-24d1-4f62-bb32-816309e8e131 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:24.047014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c11f83-affa-43f0-9199-fcefe47ac201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:24.047895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c11f83-affa-43f0-9199-fcefe47ac201 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:24.048030 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c11f83-affa-43f0-9199-fcefe47ac201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:24.048226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c11f83-affa-43f0-9199-fcefe47ac201 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:24.048982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c11f83-affa-43f0-9199-fcefe47ac201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:24.050565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c11f83-affa-43f0-9199-fcefe47ac201 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:24.060881 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1258/2516] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:24.065267 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe55765-1ae5-4f4b-a48f-04056cff38eb admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:24.075116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe55765-1ae5-4f4b-a48f-04056cff38eb admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:24.079959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc18a8-6b92-4756-8b16-9a4f2e961935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:24.080676 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc18a8-6b92-4756-8b16-9a4f2e961935 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:24.080963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc18a8-6b92-4756-8b16-9a4f2e961935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:24.081239 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc18a8-6b92-4756-8b16-9a4f2e961935 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:24.082024 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc18a8-6b92-4756-8b16-9a4f2e961935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:24.083626 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc18a8-6b92-4756-8b16-9a4f2e961935 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:24.094875 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1259/2517] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:24.098824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c11f83-affa-43f0-9199-fcefe47ac201 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:24.109321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c11f83-affa-43f0-9199-fcefe47ac201 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:24.114133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c894324-6b12-4c1f-b40e-c80873c0a23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:24.115011 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c894324-6b12-4c1f-b40e-c80873c0a23c admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:24.115299 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c894324-6b12-4c1f-b40e-c80873c0a23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:24.115580 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c894324-6b12-4c1f-b40e-c80873c0a23c admin admin] PATH_INFO: `/v3/projects/f4003caa05e1441591a307083ef56947/users/793ff798c7604336aa0b4a1401f92c15/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:24.118789 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:24.132578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c894324-6b12-4c1f-b40e-c80873c0a23c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4003caa05e1441591a307083ef56947, user_id=793ff798c7604336aa0b4a1401f92c15, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:24.135252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c894324-6b12-4c1f-b40e-c80873c0a23c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:24.148027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1259/2518] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:24 2022] PUT /identity/v3/projects/f4003caa05e1441591a307083ef56947/users/793ff798c7604336aa0b4a1401f92c15/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:24.159791 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f875386-a167-43a5-897f-1b319207507a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:24.160266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f875386-a167-43a5-897f-1b319207507a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:24.160416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f875386-a167-43a5-897f-1b319207507a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:24.188925 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1f875386-a167-43a5-897f-1b319207507a None None] MFA Rules not processed for user `793ff798c7604336aa0b4a1401f92c15`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:24.190578 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1f875386-a167-43a5-897f-1b319207507a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:24.255978 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1260/2519] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:24.260419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c894324-6b12-4c1f-b40e-c80873c0a23c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:24.322615 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c894324-6b12-4c1f-b40e-c80873c0a23c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:24.327274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c4115-7ccc-450f-a86b-5b6bd75e10e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:24.327935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c4115-7ccc-450f-a86b-5b6bd75e10e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:24.328223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c4115-7ccc-450f-a86b-5b6bd75e10e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:24.328538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c4115-7ccc-450f-a86b-5b6bd75e10e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:24.329386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c4115-7ccc-450f-a86b-5b6bd75e10e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:24.331195 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c4115-7ccc-450f-a86b-5b6bd75e10e2 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:24.359106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1260/2520] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:24 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:30:24.363228 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f875386-a167-43a5-897f-1b319207507a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:24.375261 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f875386-a167-43a5-897f-1b319207507a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:24.431799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f24f8-fa43-4092-888d-fa597df54022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:24.432517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f24f8-fa43-4092-888d-fa597df54022 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:24.432799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f24f8-fa43-4092-888d-fa597df54022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:24.433075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f24f8-fa43-4092-888d-fa597df54022 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:24.433796 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f24f8-fa43-4092-888d-fa597df54022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:24.435721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f24f8-fa43-4092-888d-fa597df54022 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:24.456980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1261/2521] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:24 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:24.460925 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c4115-7ccc-450f-a86b-5b6bd75e10e2 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:24.472086 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c4115-7ccc-450f-a86b-5b6bd75e10e2 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:24.476732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008717e8-e79b-4b01-9543-7325f1fd2746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:24.477392 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008717e8-e79b-4b01-9543-7325f1fd2746 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:24.477676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008717e8-e79b-4b01-9543-7325f1fd2746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:24.477982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008717e8-e79b-4b01-9543-7325f1fd2746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:24.478795 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008717e8-e79b-4b01-9543-7325f1fd2746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:24.480407 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008717e8-e79b-4b01-9543-7325f1fd2746 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:24.490895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1261/2522] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:24.494866 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f24f8-fa43-4092-888d-fa597df54022 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:24.505041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f24f8-fa43-4092-888d-fa597df54022 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:24.509818 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5304e3a-794f-4685-b306-478324391fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:24.510416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5304e3a-794f-4685-b306-478324391fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:24.510697 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5304e3a-794f-4685-b306-478324391fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:24.510993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5304e3a-794f-4685-b306-478324391fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:24.511725 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5304e3a-794f-4685-b306-478324391fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:24.513843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5304e3a-794f-4685-b306-478324391fd4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:24.523826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1262/2523] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:30:24.528017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008717e8-e79b-4b01-9543-7325f1fd2746 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:24.538823 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008717e8-e79b-4b01-9543-7325f1fd2746 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:24.543464 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e10d56-f9a5-4634-99cc-7dc3c6456ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:24.544425 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e10d56-f9a5-4634-99cc-7dc3c6456ed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:24.544705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e10d56-f9a5-4634-99cc-7dc3c6456ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:24.544979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e10d56-f9a5-4634-99cc-7dc3c6456ed6 admin admin] PATH_INFO: `/v3/projects/be2407582e674aaaadb75f9c32957ab0/users/98f3dfecc88e48c1a2f6742a6ebf7163/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:24.559681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e10d56-f9a5-4634-99cc-7dc3c6456ed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2407582e674aaaadb75f9c32957ab0, user_id=98f3dfecc88e48c1a2f6742a6ebf7163, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:24.562354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e10d56-f9a5-4634-99cc-7dc3c6456ed6 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:24.576278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1262/2524] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:24 2022] PUT /identity/v3/projects/be2407582e674aaaadb75f9c32957ab0/users/98f3dfecc88e48c1a2f6742a6ebf7163/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:24.586495 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6418b0-efa3-40bd-89e3-c86385e41047 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:24.586807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6418b0-efa3-40bd-89e3-c86385e41047 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:24.587089 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6418b0-efa3-40bd-89e3-c86385e41047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:24.613511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3a6418b0-efa3-40bd-89e3-c86385e41047 None None] MFA Rules not processed for user `98f3dfecc88e48c1a2f6742a6ebf7163`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:24.614731 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3a6418b0-efa3-40bd-89e3-c86385e41047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:24.619010 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:24.681100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1263/2525] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:24.685728 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e10d56-f9a5-4634-99cc-7dc3c6456ed6 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:24.746892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e10d56-f9a5-4634-99cc-7dc3c6456ed6 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:24.751465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c723a6b-6918-4198-b624-4e1bd565f14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:24.752114 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c723a6b-6918-4198-b624-4e1bd565f14c admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:24.752437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c723a6b-6918-4198-b624-4e1bd565f14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:24.752713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c723a6b-6918-4198-b624-4e1bd565f14c admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:24.753409 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c723a6b-6918-4198-b624-4e1bd565f14c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:24.755310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c723a6b-6918-4198-b624-4e1bd565f14c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:24.781263 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1263/2526] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:24 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:24.785472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6418b0-efa3-40bd-89e3-c86385e41047 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:24.796330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6418b0-efa3-40bd-89e3-c86385e41047 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:24.852124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513288cd-a068-459a-a28b-cd4b20b4af09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:24.852819 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513288cd-a068-459a-a28b-cd4b20b4af09 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:24.853100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513288cd-a068-459a-a28b-cd4b20b4af09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:24.853370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513288cd-a068-459a-a28b-cd4b20b4af09 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:24.854083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513288cd-a068-459a-a28b-cd4b20b4af09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:24.856006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513288cd-a068-459a-a28b-cd4b20b4af09 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:24.875892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1264/2527] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:24 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:24.879918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c723a6b-6918-4198-b624-4e1bd565f14c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:24.890834 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c723a6b-6918-4198-b624-4e1bd565f14c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:24.895527 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82deb2-0bdf-4483-9abd-40c6a10e6026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:24.896214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82deb2-0bdf-4483-9abd-40c6a10e6026 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:24.896534 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82deb2-0bdf-4483-9abd-40c6a10e6026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:24.896814 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82deb2-0bdf-4483-9abd-40c6a10e6026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:24.897539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82deb2-0bdf-4483-9abd-40c6a10e6026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:24.899131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82deb2-0bdf-4483-9abd-40c6a10e6026 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:24.909450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1264/2528] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:24.913736 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513288cd-a068-459a-a28b-cd4b20b4af09 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:24.924256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513288cd-a068-459a-a28b-cd4b20b4af09 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:24.928831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f80ba-4202-4dd4-bafe-b1b3d7afa676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:24.929706 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f80ba-4202-4dd4-bafe-b1b3d7afa676 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:24.929987 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f80ba-4202-4dd4-bafe-b1b3d7afa676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:24.930256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f80ba-4202-4dd4-bafe-b1b3d7afa676 admin admin] PATH_INFO: `/v3/projects/d99168901cf84cc2ba2662b05f70c8fd/users/3fd9a1da5d624ff683f145f8dea7235c/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:24.946154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f80ba-4202-4dd4-bafe-b1b3d7afa676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d99168901cf84cc2ba2662b05f70c8fd, user_id=3fd9a1da5d624ff683f145f8dea7235c, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:24.948870 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f80ba-4202-4dd4-bafe-b1b3d7afa676 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:24.962196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1265/2529] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:24 2022] PUT /identity/v3/projects/d99168901cf84cc2ba2662b05f70c8fd/users/3fd9a1da5d624ff683f145f8dea7235c/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:24.966309 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82deb2-0bdf-4483-9abd-40c6a10e6026 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:24.978384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82deb2-0bdf-4483-9abd-40c6a10e6026 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:24.983863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:25.038777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c685e4-d477-4d3c-9e6a-14835ca57d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:25.044429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c685e4-d477-4d3c-9e6a-14835ca57d31 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:25.044429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c685e4-d477-4d3c-9e6a-14835ca57d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:25.044429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c685e4-d477-4d3c-9e6a-14835ca57d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:25.044429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c685e4-d477-4d3c-9e6a-14835ca57d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:25.044429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c685e4-d477-4d3c-9e6a-14835ca57d31 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:25.053291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1265/2530] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:25.057729 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f80ba-4202-4dd4-bafe-b1b3d7afa676 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:25.069336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f80ba-4202-4dd4-bafe-b1b3d7afa676 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:25.076502 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4fb26-2070-4340-8f5f-dcd528d435fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:25.077431 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4fb26-2070-4340-8f5f-dcd528d435fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:25.077840 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4fb26-2070-4340-8f5f-dcd528d435fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:25.078198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4fb26-2070-4340-8f5f-dcd528d435fc admin admin] PATH_INFO: `/v3/projects/d99168901cf84cc2ba2662b05f70c8fd/users/3fd9a1da5d624ff683f145f8dea7235c/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:25.080228 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4fb26-2070-4340-8f5f-dcd528d435fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d99168901cf84cc2ba2662b05f70c8fd, user_id=3fd9a1da5d624ff683f145f8dea7235c, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:25.083306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4fb26-2070-4340-8f5f-dcd528d435fc admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:25.094659 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1266/2531] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:25 2022] PUT /identity/v3/projects/d99168901cf84cc2ba2662b05f70c8fd/users/3fd9a1da5d624ff683f145f8dea7235c/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:25.105669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084208a-cc3f-4f2c-b6ca-3e7a26608b78 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:25.105863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084208a-cc3f-4f2c-b6ca-3e7a26608b78 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:25.106215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084208a-cc3f-4f2c-b6ca-3e7a26608b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:25.136802 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-d084208a-cc3f-4f2c-b6ca-3e7a26608b78 None None] MFA Rules not processed for user `3fd9a1da5d624ff683f145f8dea7235c`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:25.138627 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d084208a-cc3f-4f2c-b6ca-3e7a26608b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:25.217045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1266/2532] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:25.227813 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4fb26-2070-4340-8f5f-dcd528d435fc admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:25.229538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-93b4fb26-2070-4340-8f5f-dcd528d435fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:25.290373 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4fb26-2070-4340-8f5f-dcd528d435fc admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:25.295716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2376997-b488-4309-954f-2493b3f900db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:25.296407 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2376997-b488-4309-954f-2493b3f900db service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:25.296714 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2376997-b488-4309-954f-2493b3f900db service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:25.296992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2376997-b488-4309-954f-2493b3f900db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:25.307127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2376997-b488-4309-954f-2493b3f900db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:25.308623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2376997-b488-4309-954f-2493b3f900db service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:25.314569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:25.320348 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1267/2533] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:25.895003 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084208a-cc3f-4f2c-b6ca-3e7a26608b78 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:25.904511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084208a-cc3f-4f2c-b6ca-3e7a26608b78 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:25.908179 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3dc8ec-6f17-4592-9ae7-d2181210ec9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:25.908872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dc8ec-6f17-4592-9ae7-d2181210ec9b service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:25.909156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dc8ec-6f17-4592-9ae7-d2181210ec9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:25.909434 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dc8ec-6f17-4592-9ae7-d2181210ec9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:25.966029 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3dc8ec-6f17-4592-9ae7-d2181210ec9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:25.967462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3dc8ec-6f17-4592-9ae7-d2181210ec9b service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:25.976582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1267/2534] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:28.003930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2376997-b488-4309-954f-2493b3f900db service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:28.005394 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c2376997-b488-4309-954f-2493b3f900db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:28.069521 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2376997-b488-4309-954f-2493b3f900db service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:28.074039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e987e-dfbb-47a6-b264-9be032c5c8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:28.074931 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e987e-dfbb-47a6-b264-9be032c5c8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:28.075216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e987e-dfbb-47a6-b264-9be032c5c8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:28.075491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e987e-dfbb-47a6-b264-9be032c5c8cf admin admin] PATH_INFO: `/v3/users/1e1c36ee96db4be3a416793b1e72e692` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:28.076702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e987e-dfbb-47a6-b264-9be032c5c8cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e1c36ee96db4be3a416793b1e72e692)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:28.078619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e987e-dfbb-47a6-b264-9be032c5c8cf admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:28.125014 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1c4e987e-dfbb-47a6-b264-9be032c5c8cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e1c36ee96db4be3a416793b1e72e692'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:28.132319 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1c4e987e-dfbb-47a6-b264-9be032c5c8cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e1c36ee96db4be3a416793b1e72e692'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:28.132962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1c4e987e-dfbb-47a6-b264-9be032c5c8cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e1c36ee96db4be3a416793b1e72e692'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:28.140279 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1c4e987e-dfbb-47a6-b264-9be032c5c8cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e1c36ee96db4be3a416793b1e72e692'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:28.152117 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1c4e987e-dfbb-47a6-b264-9be032c5c8cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e1c36ee96db4be3a416793b1e72e692'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:28.158901 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1268/2535] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:28 2022] DELETE /identity/v3/users/1e1c36ee96db4be3a416793b1e72e692 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:28.163949 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dcc83-9d2e-405f-a217-3539c60683f7 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:28.164445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dcc83-9d2e-405f-a217-3539c60683f7 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:28.164541 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dcc83-9d2e-405f-a217-3539c60683f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:28.181137 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-094dcc83-9d2e-405f-a217-3539c60683f7 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:28.182373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-094dcc83-9d2e-405f-a217-3539c60683f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:28.210043 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:28.243957 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1268/2536] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:28.253750 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e987e-dfbb-47a6-b264-9be032c5c8cf admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:28.255165 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1c4e987e-dfbb-47a6-b264-9be032c5c8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:28.312633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e987e-dfbb-47a6-b264-9be032c5c8cf admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:28.316771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c549d-8d4b-42b2-bed3-e9f82dcf1972 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:28.317421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c549d-8d4b-42b2-bed3-e9f82dcf1972 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:28.317706 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c549d-8d4b-42b2-bed3-e9f82dcf1972 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:28.317984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c549d-8d4b-42b2-bed3-e9f82dcf1972 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:28.327827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c549d-8d4b-42b2-bed3-e9f82dcf1972 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:28.329293 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c549d-8d4b-42b2-bed3-e9f82dcf1972 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:28.338993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1269/2537] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:28.613570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dcc83-9d2e-405f-a217-3539c60683f7 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:28.615896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-094dcc83-9d2e-405f-a217-3539c60683f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:28.642664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dcc83-9d2e-405f-a217-3539c60683f7 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:28.648250 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63057446-c311-430d-b673-4d7de2d2792c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:28.649146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63057446-c311-430d-b673-4d7de2d2792c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:28.649433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63057446-c311-430d-b673-4d7de2d2792c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:28.649709 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63057446-c311-430d-b673-4d7de2d2792c admin admin] PATH_INFO: `/v3/projects/bc7e50372ced4f049a19b02fb90670fb` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:28.650902 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63057446-c311-430d-b673-4d7de2d2792c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc7e50372ced4f049a19b02fb90670fb)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:28.652894 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63057446-c311-430d-b673-4d7de2d2792c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:28.707506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-63057446-c311-430d-b673-4d7de2d2792c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc7e50372ced4f049a19b02fb90670fb', 'request_id': 'req-63057446-c311-430d-b673-4d7de2d2792c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:28.715723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-63057446-c311-430d-b673-4d7de2d2792c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc7e50372ced4f049a19b02fb90670fb', 'request_id': 'req-63057446-c311-430d-b673-4d7de2d2792c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:28.726037 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-63057446-c311-430d-b673-4d7de2d2792c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc7e50372ced4f049a19b02fb90670fb', 'request_id': 'req-63057446-c311-430d-b673-4d7de2d2792c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:28.737726 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-63057446-c311-430d-b673-4d7de2d2792c admin admin] The token cache is being invalidate because project bc7e50372ced4f049a19b02fb90670fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:28.738142 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-63057446-c311-430d-b673-4d7de2d2792c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:28.738986 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-63057446-c311-430d-b673-4d7de2d2792c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:28.740961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1269/2538] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:28 2022] DELETE /identity/v3/projects/bc7e50372ced4f049a19b02fb90670fb => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:28.745051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c549d-8d4b-42b2-bed3-e9f82dcf1972 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:28.755500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c549d-8d4b-42b2-bed3-e9f82dcf1972 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:28.781821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:28.822928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25279ea-d86a-471b-abce-e52d6e4fabd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:28.823799 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25279ea-d86a-471b-abce-e52d6e4fabd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:28.824081 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25279ea-d86a-471b-abce-e52d6e4fabd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:28.824393 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25279ea-d86a-471b-abce-e52d6e4fabd9 admin admin] PATH_INFO: `/v3/users/b11c73f23b664d9db1bf6c34384926d0` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:28.826564 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25279ea-d86a-471b-abce-e52d6e4fabd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b11c73f23b664d9db1bf6c34384926d0)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:28.828620 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25279ea-d86a-471b-abce-e52d6e4fabd9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:28.874477 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f25279ea-d86a-471b-abce-e52d6e4fabd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b11c73f23b664d9db1bf6c34384926d0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:28.881594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f25279ea-d86a-471b-abce-e52d6e4fabd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b11c73f23b664d9db1bf6c34384926d0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:28.882200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f25279ea-d86a-471b-abce-e52d6e4fabd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b11c73f23b664d9db1bf6c34384926d0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:28.889592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f25279ea-d86a-471b-abce-e52d6e4fabd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b11c73f23b664d9db1bf6c34384926d0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:28.901023 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f25279ea-d86a-471b-abce-e52d6e4fabd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b11c73f23b664d9db1bf6c34384926d0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:28.906802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1270/2539] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:28 2022] DELETE /identity/v3/users/b11c73f23b664d9db1bf6c34384926d0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:29.186290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63057446-c311-430d-b673-4d7de2d2792c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:29.188308 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-63057446-c311-430d-b673-4d7de2d2792c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:29.205367 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63057446-c311-430d-b673-4d7de2d2792c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:29.264419 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a6fca-77be-4207-9510-777ee7e7a6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:29.265279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a6fca-77be-4207-9510-777ee7e7a6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:29.265567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a6fca-77be-4207-9510-777ee7e7a6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:29.265847 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a6fca-77be-4207-9510-777ee7e7a6e0 admin admin] PATH_INFO: `/v3/projects/c4a676566cad414d998558f439d88ec2` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:29.267044 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a6fca-77be-4207-9510-777ee7e7a6e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4a676566cad414d998558f439d88ec2)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:29.269056 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a6fca-77be-4207-9510-777ee7e7a6e0 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:29.315644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ec9a6fca-77be-4207-9510-777ee7e7a6e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4a676566cad414d998558f439d88ec2', 'request_id': 'req-ec9a6fca-77be-4207-9510-777ee7e7a6e0'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:29.319463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:29.322543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ec9a6fca-77be-4207-9510-777ee7e7a6e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4a676566cad414d998558f439d88ec2', 'request_id': 'req-ec9a6fca-77be-4207-9510-777ee7e7a6e0'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:29.329952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ec9a6fca-77be-4207-9510-777ee7e7a6e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4a676566cad414d998558f439d88ec2', 'request_id': 'req-ec9a6fca-77be-4207-9510-777ee7e7a6e0'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:29.341422 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ec9a6fca-77be-4207-9510-777ee7e7a6e0 admin admin] The token cache is being invalidate because project c4a676566cad414d998558f439d88ec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:29.341711 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ec9a6fca-77be-4207-9510-777ee7e7a6e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:29.342418 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ec9a6fca-77be-4207-9510-777ee7e7a6e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:29.343953 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1270/2540] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:29 2022] DELETE /identity/v3/projects/c4a676566cad414d998558f439d88ec2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:30:29.348903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be89cd-fa77-4a88-aa04-94eb36542bc5 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:29.349380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be89cd-fa77-4a88-aa04-94eb36542bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:29.349491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be89cd-fa77-4a88-aa04-94eb36542bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:29.367505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b0be89cd-fa77-4a88-aa04-94eb36542bc5 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:29.368842 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b0be89cd-fa77-4a88-aa04-94eb36542bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:29.438081 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1271/2541] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:29.446118 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46ce78-d4b6-4997-bfe0-ac7b2d1eb903 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:29.446434 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46ce78-d4b6-4997-bfe0-ac7b2d1eb903 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:29.446715 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46ce78-d4b6-4997-bfe0-ac7b2d1eb903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:29.464846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-0f46ce78-d4b6-4997-bfe0-ac7b2d1eb903 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:29.466531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0f46ce78-d4b6-4997-bfe0-ac7b2d1eb903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:29.474786 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1271/2542] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:29.479107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be89cd-fa77-4a88-aa04-94eb36542bc5 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:29.489761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be89cd-fa77-4a88-aa04-94eb36542bc5 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:29.494357 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412484f1-db8a-4c3a-a529-c35aeb94ff21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:29.495005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412484f1-db8a-4c3a-a529-c35aeb94ff21 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:29.495296 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412484f1-db8a-4c3a-a529-c35aeb94ff21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:29.495574 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412484f1-db8a-4c3a-a529-c35aeb94ff21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:29.496325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412484f1-db8a-4c3a-a529-c35aeb94ff21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:30:29.496671 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412484f1-db8a-4c3a-a529-c35aeb94ff21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:29.498274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412484f1-db8a-4c3a-a529-c35aeb94ff21 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:29.509945 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1272/2543] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:30:29.514156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46ce78-d4b6-4997-bfe0-ac7b2d1eb903 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:29.523510 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46ce78-d4b6-4997-bfe0-ac7b2d1eb903 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:29.528657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fc26f-a741-420a-baa9-bab0a22b4b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:29.529295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fc26f-a741-420a-baa9-bab0a22b4b29 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:29.529573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fc26f-a741-420a-baa9-bab0a22b4b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:29.529844 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fc26f-a741-420a-baa9-bab0a22b4b29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:29.530566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fc26f-a741-420a-baa9-bab0a22b4b29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:29.532463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fc26f-a741-420a-baa9-bab0a22b4b29 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:29.558490 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1272/2544] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:29 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:29.563061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412484f1-db8a-4c3a-a529-c35aeb94ff21 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:29.573159 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412484f1-db8a-4c3a-a529-c35aeb94ff21 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:29.599701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:29.630814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afd3e0-6403-4b5b-91a1-f20e7e5f1058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:29.631481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afd3e0-6403-4b5b-91a1-f20e7e5f1058 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:29.631762 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afd3e0-6403-4b5b-91a1-f20e7e5f1058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:29.632037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afd3e0-6403-4b5b-91a1-f20e7e5f1058 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:29.632766 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afd3e0-6403-4b5b-91a1-f20e7e5f1058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:29.634684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afd3e0-6403-4b5b-91a1-f20e7e5f1058 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:29.653939 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1273/2545] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:29 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:30:29.658090 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fc26f-a741-420a-baa9-bab0a22b4b29 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:29.668352 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fc26f-a741-420a-baa9-bab0a22b4b29 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:29.673590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d24e8e-a3ba-4270-8609-f06f389c60bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:29.674247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d24e8e-a3ba-4270-8609-f06f389c60bf admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:29.674533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d24e8e-a3ba-4270-8609-f06f389c60bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:29.674813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d24e8e-a3ba-4270-8609-f06f389c60bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:29.675540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d24e8e-a3ba-4270-8609-f06f389c60bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:29.677155 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d24e8e-a3ba-4270-8609-f06f389c60bf admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:29.688076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1273/2546] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:30:29.692523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afd3e0-6403-4b5b-91a1-f20e7e5f1058 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:29.702602 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afd3e0-6403-4b5b-91a1-f20e7e5f1058 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:29.707083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a38b0-4447-4515-aa49-4ca33cb80b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:29.707729 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a38b0-4447-4515-aa49-4ca33cb80b0a admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:29.708011 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a38b0-4447-4515-aa49-4ca33cb80b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:29.708340 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a38b0-4447-4515-aa49-4ca33cb80b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:29.709048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a38b0-4447-4515-aa49-4ca33cb80b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:29.710647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a38b0-4447-4515-aa49-4ca33cb80b0a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:29.720919 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1274/2547] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:29.725034 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d24e8e-a3ba-4270-8609-f06f389c60bf admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:29.735575 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d24e8e-a3ba-4270-8609-f06f389c60bf admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:29.740158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafc27b-8a75-48cb-b9c3-32c8af574f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:29.741071 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafc27b-8a75-48cb-b9c3-32c8af574f8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:29.741359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafc27b-8a75-48cb-b9c3-32c8af574f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:29.741688 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafc27b-8a75-48cb-b9c3-32c8af574f8c admin admin] PATH_INFO: `/v3/projects/9a2a292835084b669f9ad97d34308580/users/9a16556c1ade49b0a25b350fa6dc797a/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:29.756633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafc27b-8a75-48cb-b9c3-32c8af574f8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a2a292835084b669f9ad97d34308580, user_id=9a16556c1ade49b0a25b350fa6dc797a, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:29.759285 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafc27b-8a75-48cb-b9c3-32c8af574f8c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:29.771959 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1274/2548] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:29 2022] PUT /identity/v3/projects/9a2a292835084b669f9ad97d34308580/users/9a16556c1ade49b0a25b350fa6dc797a/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:30:29.783129 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b37e1c-9490-4bf0-bc62-616af6e1ecb4 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:29.783423 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b37e1c-9490-4bf0-bc62-616af6e1ecb4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:29.783889 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b37e1c-9490-4bf0-bc62-616af6e1ecb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:29.812803 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d1b37e1c-9490-4bf0-bc62-616af6e1ecb4 None None] MFA Rules not processed for user `9a16556c1ade49b0a25b350fa6dc797a`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:29.814002 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d1b37e1c-9490-4bf0-bc62-616af6e1ecb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:29.888402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1275/2549] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:29.892680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafc27b-8a75-48cb-b9c3-32c8af574f8c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:29.957483 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafc27b-8a75-48cb-b9c3-32c8af574f8c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:29.962140 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73562db-2acb-46df-925f-b629363bf23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:29.962818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73562db-2acb-46df-925f-b629363bf23b admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:29.963084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73562db-2acb-46df-925f-b629363bf23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:29.963363 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73562db-2acb-46df-925f-b629363bf23b admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:29.964094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73562db-2acb-46df-925f-b629363bf23b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:29.966570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73562db-2acb-46df-925f-b629363bf23b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:29.983680 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b37e1c-9490-4bf0-bc62-616af6e1ecb4 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:29.985659 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d1b37e1c-9490-4bf0-bc62-616af6e1ecb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:30.000067 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1275/2550] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:29 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:30.003801 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73562db-2acb-46df-925f-b629363bf23b admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:30.047575 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b37e1c-9490-4bf0-bc62-616af6e1ecb4 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:30.067724 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73562db-2acb-46df-925f-b629363bf23b admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:30.072595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad64e0-1424-438c-a390-ece45498ec11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:30.073371 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad64e0-1424-438c-a390-ece45498ec11 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:30.073658 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad64e0-1424-438c-a390-ece45498ec11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:30.073938 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad64e0-1424-438c-a390-ece45498ec11 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:30.074637 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad64e0-1424-438c-a390-ece45498ec11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:30.076657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad64e0-1424-438c-a390-ece45498ec11 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:30.085479 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:30.098264 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b43eeb-f54e-4fa2-a9b2-25adc3a98b4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:30.098989 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b43eeb-f54e-4fa2-a9b2-25adc3a98b4a service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:30.099278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b43eeb-f54e-4fa2-a9b2-25adc3a98b4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:30.099571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b43eeb-f54e-4fa2-a9b2-25adc3a98b4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:30.100075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1276/2551] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:30 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:30:30.102272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d6b43eeb-f54e-4fa2-a9b2-25adc3a98b4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:30.103627 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad64e0-1424-438c-a390-ece45498ec11 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:30.109344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:30.120665 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad64e0-1424-438c-a390-ece45498ec11 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:30.125477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54237e-877a-40c9-b4b6-562b88c1f3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:30.126145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54237e-877a-40c9-b4b6-562b88c1f3fb admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:30.126432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54237e-877a-40c9-b4b6-562b88c1f3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:30.126708 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54237e-877a-40c9-b4b6-562b88c1f3fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:30.127471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54237e-877a-40c9-b4b6-562b88c1f3fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:30.129074 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54237e-877a-40c9-b4b6-562b88c1f3fb admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:30.142286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1277/2552] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:30:30.142286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54237e-877a-40c9-b4b6-562b88c1f3fb admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:30.142286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5a54237e-877a-40c9-b4b6-562b88c1f3fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:30.153917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54237e-877a-40c9-b4b6-562b88c1f3fb admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:30.157593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea6fe4-46ad-4cf5-959a-a7f8c7f6e191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:30.158224 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea6fe4-46ad-4cf5-959a-a7f8c7f6e191 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:30.158507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea6fe4-46ad-4cf5-959a-a7f8c7f6e191 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:30.158782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea6fe4-46ad-4cf5-959a-a7f8c7f6e191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:30.160976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-cfea6fe4-46ad-4cf5-959a-a7f8c7f6e191 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:30.170689 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b43eeb-f54e-4fa2-a9b2-25adc3a98b4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:30.172337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b43eeb-f54e-4fa2-a9b2-25adc3a98b4a service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:30.181474 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1276/2553] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:30:30.182258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b43eeb-f54e-4fa2-a9b2-25adc3a98b4a service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:30.193043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b43eeb-f54e-4fa2-a9b2-25adc3a98b4a service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:30.198652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876c8c8-360a-4ad3-b956-ca1b77c38cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:30.199538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876c8c8-360a-4ad3-b956-ca1b77c38cff admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:30.199852 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876c8c8-360a-4ad3-b956-ca1b77c38cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:30.200130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876c8c8-360a-4ad3-b956-ca1b77c38cff admin admin] PATH_INFO: `/v3/projects/0a47b204a0364c1a841059859eedc8d9/users/e778d466cbaa4701af41d5a5094fefc6/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:30.217734 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876c8c8-360a-4ad3-b956-ca1b77c38cff admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a47b204a0364c1a841059859eedc8d9, user_id=e778d466cbaa4701af41d5a5094fefc6, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:30.220826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876c8c8-360a-4ad3-b956-ca1b77c38cff admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:30.226249 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea6fe4-46ad-4cf5-959a-a7f8c7f6e191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:30.227886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea6fe4-46ad-4cf5-959a-a7f8c7f6e191 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:30.241105 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1277/2554] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:30 2022] PUT /identity/v3/projects/0a47b204a0364c1a841059859eedc8d9/users/e778d466cbaa4701af41d5a5094fefc6/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:30.241291 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1278/2555] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:30.242114 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876c8c8-360a-4ad3-b956-ca1b77c38cff admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:30.312826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876c8c8-360a-4ad3-b956-ca1b77c38cff admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:30.319041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6643f-5404-4742-8f91-8fb515c4fd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:30.319835 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6643f-5404-4742-8f91-8fb515c4fd18 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:30.320228 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6643f-5404-4742-8f91-8fb515c4fd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:30.320681 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6643f-5404-4742-8f91-8fb515c4fd18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:30.321860 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6643f-5404-4742-8f91-8fb515c4fd18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:30.324085 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6643f-5404-4742-8f91-8fb515c4fd18 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:30.346829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1278/2556] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:30:30.348119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea6fe4-46ad-4cf5-959a-a7f8c7f6e191 service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:30.361841 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea6fe4-46ad-4cf5-959a-a7f8c7f6e191 service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:30.366927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4fea9e-bd94-4123-b316-3427787195b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:30.367828 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fea9e-bd94-4123-b316-3427787195b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:30.368119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fea9e-bd94-4123-b316-3427787195b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:30.368431 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fea9e-bd94-4123-b316-3427787195b4 admin admin] PATH_INFO: `/v3/projects/0a47b204a0364c1a841059859eedc8d9/users/e778d466cbaa4701af41d5a5094fefc6/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:30.371287 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4fea9e-bd94-4123-b316-3427787195b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a47b204a0364c1a841059859eedc8d9, user_id=e778d466cbaa4701af41d5a5094fefc6, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:30.374017 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4fea9e-bd94-4123-b316-3427787195b4 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:30.388655 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1279/2557] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:30 2022] PUT /identity/v3/projects/0a47b204a0364c1a841059859eedc8d9/users/e778d466cbaa4701af41d5a5094fefc6/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:30:30.401035 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdcbf6-0e1f-41b9-b3af-13bd090d784e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:30.401504 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdcbf6-0e1f-41b9-b3af-13bd090d784e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:30.401955 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdcbf6-0e1f-41b9-b3af-13bd090d784e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:30.457307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8ebdcbf6-0e1f-41b9-b3af-13bd090d784e None None] MFA Rules not processed for user `e778d466cbaa4701af41d5a5094fefc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:30.458725 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8ebdcbf6-0e1f-41b9-b3af-13bd090d784e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:30.541830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1279/2558] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:30 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:30.553496 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4fea9e-bd94-4123-b316-3427787195b4 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:30.620657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4fea9e-bd94-4123-b316-3427787195b4 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:30.624248 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da3697-2fd4-4f05-b444-25eae5667cf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:30.624922 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da3697-2fd4-4f05-b444-25eae5667cf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:30.625212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da3697-2fd4-4f05-b444-25eae5667cf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:30.625493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da3697-2fd4-4f05-b444-25eae5667cf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:30.634874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da3697-2fd4-4f05-b444-25eae5667cf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:30.636304 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da3697-2fd4-4f05-b444-25eae5667cf1 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:30.647227 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1280/2559] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:30 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:32.099400 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdcbf6-0e1f-41b9-b3af-13bd090d784e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:32.105011 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:32.167671 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdcbf6-0e1f-41b9-b3af-13bd090d784e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:32.172398 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb41810-21c2-4545-9648-481122b492bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:32.173303 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb41810-21c2-4545-9648-481122b492bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:32.173572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb41810-21c2-4545-9648-481122b492bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:32.173850 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb41810-21c2-4545-9648-481122b492bf admin admin] PATH_INFO: `/v3/users/9a16556c1ade49b0a25b350fa6dc797a` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:32.175057 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb41810-21c2-4545-9648-481122b492bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a16556c1ade49b0a25b350fa6dc797a)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:32.177056 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb41810-21c2-4545-9648-481122b492bf admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:32.227883 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0bb41810-21c2-4545-9648-481122b492bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a16556c1ade49b0a25b350fa6dc797a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:32.237054 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0bb41810-21c2-4545-9648-481122b492bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a16556c1ade49b0a25b350fa6dc797a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:32.237674 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0bb41810-21c2-4545-9648-481122b492bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a16556c1ade49b0a25b350fa6dc797a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:32.245140 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0bb41810-21c2-4545-9648-481122b492bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a16556c1ade49b0a25b350fa6dc797a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:32.257633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0bb41810-21c2-4545-9648-481122b492bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a16556c1ade49b0a25b350fa6dc797a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:32.264215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1280/2560] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:32 2022] DELETE /identity/v3/users/9a16556c1ade49b0a25b350fa6dc797a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:32.268820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da73aa-da7d-4b43-b4b6-430c9875b227 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:32.269301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da73aa-da7d-4b43-b4b6-430c9875b227 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:32.269405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da73aa-da7d-4b43-b4b6-430c9875b227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:32.287966 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-48da73aa-da7d-4b43-b4b6-430c9875b227 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:32.289295 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-48da73aa-da7d-4b43-b4b6-430c9875b227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:32.358585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1281/2561] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:32.368247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb41810-21c2-4545-9648-481122b492bf admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:32.369700 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0bb41810-21c2-4545-9648-481122b492bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:32.429900 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb41810-21c2-4545-9648-481122b492bf admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:32.433573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2082911-2c90-471f-9786-64c80db117f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:32.434710 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2082911-2c90-471f-9786-64c80db117f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:32.435337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2082911-2c90-471f-9786-64c80db117f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:32.435781 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2082911-2c90-471f-9786-64c80db117f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:32.446776 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2082911-2c90-471f-9786-64c80db117f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:32.448201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2082911-2c90-471f-9786-64c80db117f0 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:32.459296 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1281/2562] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:32.741600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da73aa-da7d-4b43-b4b6-430c9875b227 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:32.743056 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-48da73aa-da7d-4b43-b4b6-430c9875b227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:32.757740 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da73aa-da7d-4b43-b4b6-430c9875b227 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:32.762990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae845fc-31c8-4e1b-9d08-720416f3d16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:32.763897 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae845fc-31c8-4e1b-9d08-720416f3d16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:32.764256 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae845fc-31c8-4e1b-9d08-720416f3d16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:32.764472 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae845fc-31c8-4e1b-9d08-720416f3d16e admin admin] PATH_INFO: `/v3/projects/9a2a292835084b669f9ad97d34308580` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:32.765729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae845fc-31c8-4e1b-9d08-720416f3d16e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a2a292835084b669f9ad97d34308580)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:32.767914 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae845fc-31c8-4e1b-9d08-720416f3d16e admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:32.775065 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:32.821593 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6ae845fc-31c8-4e1b-9d08-720416f3d16e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a2a292835084b669f9ad97d34308580', 'request_id': 'req-6ae845fc-31c8-4e1b-9d08-720416f3d16e'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:32.829583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6ae845fc-31c8-4e1b-9d08-720416f3d16e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a2a292835084b669f9ad97d34308580', 'request_id': 'req-6ae845fc-31c8-4e1b-9d08-720416f3d16e'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:32.835688 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6ae845fc-31c8-4e1b-9d08-720416f3d16e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a2a292835084b669f9ad97d34308580', 'request_id': 'req-6ae845fc-31c8-4e1b-9d08-720416f3d16e'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:32.847553 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6ae845fc-31c8-4e1b-9d08-720416f3d16e admin admin] The token cache is being invalidate because project 9a2a292835084b669f9ad97d34308580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:32.847839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6ae845fc-31c8-4e1b-9d08-720416f3d16e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:32.848537 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6ae845fc-31c8-4e1b-9d08-720416f3d16e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:32.850060 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1282/2563] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:32 2022] DELETE /identity/v3/projects/9a2a292835084b669f9ad97d34308580 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:32.854098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2082911-2c90-471f-9786-64c80db117f0 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:32.864403 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2082911-2c90-471f-9786-64c80db117f0 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:32.888976 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:32.926729 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd27db-9631-4e39-be29-688fa956cc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:32.927579 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd27db-9631-4e39-be29-688fa956cc4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:32.927865 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd27db-9631-4e39-be29-688fa956cc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:32.928189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd27db-9631-4e39-be29-688fa956cc4c admin admin] PATH_INFO: `/v3/users/e778d466cbaa4701af41d5a5094fefc6` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:32.929530 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd27db-9631-4e39-be29-688fa956cc4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e778d466cbaa4701af41d5a5094fefc6)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:32.931562 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd27db-9631-4e39-be29-688fa956cc4c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:32.982665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0afd27db-9631-4e39-be29-688fa956cc4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e778d466cbaa4701af41d5a5094fefc6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:32.991130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0afd27db-9631-4e39-be29-688fa956cc4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e778d466cbaa4701af41d5a5094fefc6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:32.991829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0afd27db-9631-4e39-be29-688fa956cc4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e778d466cbaa4701af41d5a5094fefc6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:32.999189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0afd27db-9631-4e39-be29-688fa956cc4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e778d466cbaa4701af41d5a5094fefc6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:33.011243 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0afd27db-9631-4e39-be29-688fa956cc4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e778d466cbaa4701af41d5a5094fefc6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:33.018201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1282/2564] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:32 2022] DELETE /identity/v3/users/e778d466cbaa4701af41d5a5094fefc6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:33.321086 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae845fc-31c8-4e1b-9d08-720416f3d16e admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:33.322182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6ae845fc-31c8-4e1b-9d08-720416f3d16e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:33.337820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae845fc-31c8-4e1b-9d08-720416f3d16e admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:33.406373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9effa0e9-0f0f-4297-82a7-e91702b33741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:33.407736 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effa0e9-0f0f-4297-82a7-e91702b33741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:33.408027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effa0e9-0f0f-4297-82a7-e91702b33741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:33.408506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effa0e9-0f0f-4297-82a7-e91702b33741 admin admin] PATH_INFO: `/v3/projects/0a47b204a0364c1a841059859eedc8d9` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:33.409616 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9effa0e9-0f0f-4297-82a7-e91702b33741 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a47b204a0364c1a841059859eedc8d9)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:33.411531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9effa0e9-0f0f-4297-82a7-e91702b33741 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:33.461143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9effa0e9-0f0f-4297-82a7-e91702b33741 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a47b204a0364c1a841059859eedc8d9', 'request_id': 'req-9effa0e9-0f0f-4297-82a7-e91702b33741'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:33.470271 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9effa0e9-0f0f-4297-82a7-e91702b33741 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a47b204a0364c1a841059859eedc8d9', 'request_id': 'req-9effa0e9-0f0f-4297-82a7-e91702b33741'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:33.478630 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9effa0e9-0f0f-4297-82a7-e91702b33741 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a47b204a0364c1a841059859eedc8d9', 'request_id': 'req-9effa0e9-0f0f-4297-82a7-e91702b33741'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:33.490747 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9effa0e9-0f0f-4297-82a7-e91702b33741 admin admin] The token cache is being invalidate because project 0a47b204a0364c1a841059859eedc8d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:33.491035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9effa0e9-0f0f-4297-82a7-e91702b33741 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:33.491689 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9effa0e9-0f0f-4297-82a7-e91702b33741 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:33.493265 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1283/2565] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:33 2022] DELETE /identity/v3/projects/0a47b204a0364c1a841059859eedc8d9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:33.499036 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b323db-dadc-4b50-9067-16c395eabf83 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:33.499540 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b323db-dadc-4b50-9067-16c395eabf83 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:33.499624 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b323db-dadc-4b50-9067-16c395eabf83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:33.520046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f0b323db-dadc-4b50-9067-16c395eabf83 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:33.521376 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f0b323db-dadc-4b50-9067-16c395eabf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:33.594336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1283/2566] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:33.606396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f3212-f1e2-43b0-9c80-a60eddf90b1e None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:33.606839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f3212-f1e2-43b0-9c80-a60eddf90b1e None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:33.607124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f3212-f1e2-43b0-9c80-a60eddf90b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:33.628238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-a31f3212-f1e2-43b0-9c80-a60eddf90b1e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:33.630912 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-a31f3212-f1e2-43b0-9c80-a60eddf90b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:33.641139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1284/2567] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:33.646228 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b323db-dadc-4b50-9067-16c395eabf83 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:33.657377 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b323db-dadc-4b50-9067-16c395eabf83 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:33.664757 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822e42d-e601-486a-86eb-4b9ea0e1c2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:33.665952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822e42d-e601-486a-86eb-4b9ea0e1c2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:33.666429 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822e42d-e601-486a-86eb-4b9ea0e1c2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:33.666886 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822e42d-e601-486a-86eb-4b9ea0e1c2e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:33.668847 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822e42d-e601-486a-86eb-4b9ea0e1c2e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:30:33.669212 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822e42d-e601-486a-86eb-4b9ea0e1c2e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:33.672133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822e42d-e601-486a-86eb-4b9ea0e1c2e0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:33.685946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1284/2568] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:30:33.691703 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f3212-f1e2-43b0-9c80-a60eddf90b1e None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:33.707173 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f3212-f1e2-43b0-9c80-a60eddf90b1e None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:33.709731 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf650bea-9938-43e0-86f1-04bee29aa2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:33.710688 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf650bea-9938-43e0-86f1-04bee29aa2cf admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:33.710975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf650bea-9938-43e0-86f1-04bee29aa2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:33.711312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf650bea-9938-43e0-86f1-04bee29aa2cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:33.712119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf650bea-9938-43e0-86f1-04bee29aa2cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:33.714205 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf650bea-9938-43e0-86f1-04bee29aa2cf admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:33.730457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:33.744327 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1285/2569] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:33 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:33.748464 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822e42d-e601-486a-86eb-4b9ea0e1c2e0 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:33.759358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822e42d-e601-486a-86eb-4b9ea0e1c2e0 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:33.795356 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:33.832722 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64b4e2-0959-4a62-acdc-5ebcfb4ef3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:33.833883 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64b4e2-0959-4a62-acdc-5ebcfb4ef3f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:33.834554 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64b4e2-0959-4a62-acdc-5ebcfb4ef3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:33.835047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64b4e2-0959-4a62-acdc-5ebcfb4ef3f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:33.836124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64b4e2-0959-4a62-acdc-5ebcfb4ef3f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:33.840649 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64b4e2-0959-4a62-acdc-5ebcfb4ef3f4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:33.867828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1285/2570] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:33 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:33.873450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf650bea-9938-43e0-86f1-04bee29aa2cf admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:33.886891 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf650bea-9938-43e0-86f1-04bee29aa2cf admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:33.893543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489cd3b8-e494-4561-9ef1-7184ba2680d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:33.894341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cd3b8-e494-4561-9ef1-7184ba2680d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:33.894629 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cd3b8-e494-4561-9ef1-7184ba2680d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:33.894910 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cd3b8-e494-4561-9ef1-7184ba2680d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:33.895682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489cd3b8-e494-4561-9ef1-7184ba2680d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:33.897332 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489cd3b8-e494-4561-9ef1-7184ba2680d7 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:33.909879 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1286/2571] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:33.913930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64b4e2-0959-4a62-acdc-5ebcfb4ef3f4 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:33.924788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64b4e2-0959-4a62-acdc-5ebcfb4ef3f4 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:33.929628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba8360-17ee-4331-a97e-5a41ef0edc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:33.930308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba8360-17ee-4331-a97e-5a41ef0edc7e admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:33.930592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba8360-17ee-4331-a97e-5a41ef0edc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:33.930868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba8360-17ee-4331-a97e-5a41ef0edc7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:33.931595 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba8360-17ee-4331-a97e-5a41ef0edc7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:33.933237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba8360-17ee-4331-a97e-5a41ef0edc7e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:33.948452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1286/2572] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:30:33.950526 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489cd3b8-e494-4561-9ef1-7184ba2680d7 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:33.960983 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489cd3b8-e494-4561-9ef1-7184ba2680d7 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:33.966002 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7326ec6-0e47-4b43-8ce1-a0e2ccd64519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:33.966834 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7326ec6-0e47-4b43-8ce1-a0e2ccd64519 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:33.967108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7326ec6-0e47-4b43-8ce1-a0e2ccd64519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:33.967380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7326ec6-0e47-4b43-8ce1-a0e2ccd64519 admin admin] PATH_INFO: `/v3/projects/68267c8e8240468cb76f7aea8c8f994c/users/81d6d182cdd742cbb34707da109c985f/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:33.985650 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7326ec6-0e47-4b43-8ce1-a0e2ccd64519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68267c8e8240468cb76f7aea8c8f994c, user_id=81d6d182cdd742cbb34707da109c985f, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:33.988753 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7326ec6-0e47-4b43-8ce1-a0e2ccd64519 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:34.003883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1287/2573] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:33 2022] PUT /identity/v3/projects/68267c8e8240468cb76f7aea8c8f994c/users/81d6d182cdd742cbb34707da109c985f/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:34.016161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e1ccf-cba1-4148-9754-b0a70084013d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:34.016481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e1ccf-cba1-4148-9754-b0a70084013d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:34.016778 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e1ccf-cba1-4148-9754-b0a70084013d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:34.047569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-181e1ccf-cba1-4148-9754-b0a70084013d None None] MFA Rules not processed for user `81d6d182cdd742cbb34707da109c985f`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:34.048919 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-181e1ccf-cba1-4148-9754-b0a70084013d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:34.131002 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1287/2574] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:34.132920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7326ec6-0e47-4b43-8ce1-a0e2ccd64519 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:34.220324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7326ec6-0e47-4b43-8ce1-a0e2ccd64519 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:34.225661 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a706711-2ff3-49af-91e4-efb96a3e2fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:34.226411 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a706711-2ff3-49af-91e4-efb96a3e2fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:34.226765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a706711-2ff3-49af-91e4-efb96a3e2fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:34.227111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a706711-2ff3-49af-91e4-efb96a3e2fe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:34.227873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a706711-2ff3-49af-91e4-efb96a3e2fe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:34.229872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a706711-2ff3-49af-91e4-efb96a3e2fe9 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:34.268337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1288/2575] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:34 2022] POST /identity/v3/projects => generated 377 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:34.270789 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e1ccf-cba1-4148-9754-b0a70084013d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:34.282109 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e1ccf-cba1-4148-9754-b0a70084013d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:34.362328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47e6d7-e57b-4b10-bcf9-ddc2228a4d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:34.363104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47e6d7-e57b-4b10-bcf9-ddc2228a4d3e admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:34.363472 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47e6d7-e57b-4b10-bcf9-ddc2228a4d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:34.363797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47e6d7-e57b-4b10-bcf9-ddc2228a4d3e admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:34.364664 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47e6d7-e57b-4b10-bcf9-ddc2228a4d3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:34.366712 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47e6d7-e57b-4b10-bcf9-ddc2228a4d3e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:34.392541 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1288/2576] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:34 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:34.398130 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a706711-2ff3-49af-91e4-efb96a3e2fe9 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:34.410929 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a706711-2ff3-49af-91e4-efb96a3e2fe9 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:34.418246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58325524-9710-4687-846c-c1ea7ba22adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:34.420001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58325524-9710-4687-846c-c1ea7ba22adf admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:34.420329 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58325524-9710-4687-846c-c1ea7ba22adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:34.420681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58325524-9710-4687-846c-c1ea7ba22adf admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:34.421572 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58325524-9710-4687-846c-c1ea7ba22adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:34.423457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58325524-9710-4687-846c-c1ea7ba22adf admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:34.436629 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1289/2577] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:34.441403 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47e6d7-e57b-4b10-bcf9-ddc2228a4d3e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:34.452779 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47e6d7-e57b-4b10-bcf9-ddc2228a4d3e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:34.458936 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d5f23-f241-4ea8-a737-3da5f733a65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:34.459716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d5f23-f241-4ea8-a737-3da5f733a65e admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:34.460037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d5f23-f241-4ea8-a737-3da5f733a65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:34.460398 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d5f23-f241-4ea8-a737-3da5f733a65e admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:34.461194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d5f23-f241-4ea8-a737-3da5f733a65e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:34.462824 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d5f23-f241-4ea8-a737-3da5f733a65e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:34.475957 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1289/2578] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:34.479991 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58325524-9710-4687-846c-c1ea7ba22adf admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:34.493252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58325524-9710-4687-846c-c1ea7ba22adf admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:34.499283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b3508-a5d1-45f8-b176-bf1932f8cbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:34.500215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b3508-a5d1-45f8-b176-bf1932f8cbcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:34.500603 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b3508-a5d1-45f8-b176-bf1932f8cbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:34.500973 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b3508-a5d1-45f8-b176-bf1932f8cbcf admin admin] PATH_INFO: `/v3/projects/ac6a55f67d174d6bb9974062231afb3c/users/5ece30e2bb834d9b85f71b519d26af41/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:34.519490 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b3508-a5d1-45f8-b176-bf1932f8cbcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac6a55f67d174d6bb9974062231afb3c, user_id=5ece30e2bb834d9b85f71b519d26af41, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:34.522253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b3508-a5d1-45f8-b176-bf1932f8cbcf admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:34.552354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1290/2579] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:34 2022] PUT /identity/v3/projects/ac6a55f67d174d6bb9974062231afb3c/users/5ece30e2bb834d9b85f71b519d26af41/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:34.556000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97403e02-9e2d-4464-b914-5a7a145ded8c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:34.556688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97403e02-9e2d-4464-b914-5a7a145ded8c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:34.556868 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97403e02-9e2d-4464-b914-5a7a145ded8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:34.586559 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-97403e02-9e2d-4464-b914-5a7a145ded8c None None] MFA Rules not processed for user `5ece30e2bb834d9b85f71b519d26af41`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:34.587889 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-97403e02-9e2d-4464-b914-5a7a145ded8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:34.598453 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:34.657118 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b3508-a5d1-45f8-b176-bf1932f8cbcf admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:34.659695 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-cd9b3508-a5d1-45f8-b176-bf1932f8cbcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:34.683889 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1290/2580] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:34.688383 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97403e02-9e2d-4464-b914-5a7a145ded8c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:34.761120 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b3508-a5d1-45f8-b176-bf1932f8cbcf admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:34.766800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2ebb3-8396-4921-9695-0a9627b9d54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:34.767785 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2ebb3-8396-4921-9695-0a9627b9d54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:34.768075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2ebb3-8396-4921-9695-0a9627b9d54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:34.768385 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2ebb3-8396-4921-9695-0a9627b9d54f admin admin] PATH_INFO: `/v3/users/793ff798c7604336aa0b4a1401f92c15` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:34.769609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2ebb3-8396-4921-9695-0a9627b9d54f admin admin] RBAC: Authorizing `identity:delete_user(user_id=793ff798c7604336aa0b4a1401f92c15)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:34.771551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2ebb3-8396-4921-9695-0a9627b9d54f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:34.775711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97403e02-9e2d-4464-b914-5a7a145ded8c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:34.781362 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefd0a0-a899-44b1-a4bb-8a187cd8db5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:34.782241 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefd0a0-a899-44b1-a4bb-8a187cd8db5c admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:34.782529 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefd0a0-a899-44b1-a4bb-8a187cd8db5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:34.782806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefd0a0-a899-44b1-a4bb-8a187cd8db5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:34.783513 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefd0a0-a899-44b1-a4bb-8a187cd8db5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:34.785499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefd0a0-a899-44b1-a4bb-8a187cd8db5c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:34.787176 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:34.829366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1291/2581] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:34 2022] POST /identity/v3/projects => generated 379 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:30:34.833602 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefd0a0-a899-44b1-a4bb-8a187cd8db5c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:34.846576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-48e2ebb3-8396-4921-9695-0a9627b9d54f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '793ff798c7604336aa0b4a1401f92c15'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:34.859977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-48e2ebb3-8396-4921-9695-0a9627b9d54f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '793ff798c7604336aa0b4a1401f92c15'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:34.878029 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-48e2ebb3-8396-4921-9695-0a9627b9d54f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '793ff798c7604336aa0b4a1401f92c15'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:34.892395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-48e2ebb3-8396-4921-9695-0a9627b9d54f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '793ff798c7604336aa0b4a1401f92c15'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:34.893963 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-48e2ebb3-8396-4921-9695-0a9627b9d54f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '793ff798c7604336aa0b4a1401f92c15'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:34.903112 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1291/2582] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:34 2022] DELETE /identity/v3/users/793ff798c7604336aa0b4a1401f92c15 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:30:34.908051 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c741e2c-c2ce-405e-85b1-7b648cc8b3d5 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:34.908618 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c741e2c-c2ce-405e-85b1-7b648cc8b3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:34.909480 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c741e2c-c2ce-405e-85b1-7b648cc8b3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:34.930435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefd0a0-a899-44b1-a4bb-8a187cd8db5c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:34.936214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4c741e2c-c2ce-405e-85b1-7b648cc8b3d5 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:34.938387 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4c741e2c-c2ce-405e-85b1-7b648cc8b3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:34.985220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:35.015155 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9652935-c2e9-495a-bb0b-b1bac44139a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:35.015866 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9652935-c2e9-495a-bb0b-b1bac44139a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:35.016151 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9652935-c2e9-495a-bb0b-b1bac44139a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:35.016600 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9652935-c2e9-495a-bb0b-b1bac44139a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:35.017392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9652935-c2e9-495a-bb0b-b1bac44139a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:35.019252 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9652935-c2e9-495a-bb0b-b1bac44139a5 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:35.030508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1292/2583] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:35.039574 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c741e2c-c2ce-405e-85b1-7b648cc8b3d5 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:35.041911 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4c741e2c-c2ce-405e-85b1-7b648cc8b3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:35.048171 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1292/2584] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:34 2022] POST /identity/v3/users => generated 296 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:30:35.052648 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9652935-c2e9-495a-bb0b-b1bac44139a5 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:35.054869 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b9652935-c2e9-495a-bb0b-b1bac44139a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:35.072942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9652935-c2e9-495a-bb0b-b1bac44139a5 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:35.082764 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e30072-371a-4634-a06a-a5b36a489608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:35.083701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e30072-371a-4634-a06a-a5b36a489608 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:35.084205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e30072-371a-4634-a06a-a5b36a489608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:35.084741 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e30072-371a-4634-a06a-a5b36a489608 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:35.085594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e30072-371a-4634-a06a-a5b36a489608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:35.087297 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e30072-371a-4634-a06a-a5b36a489608 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:35.109016 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1293/2585] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:35.110668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e30072-371a-4634-a06a-a5b36a489608 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:35.128875 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e30072-371a-4634-a06a-a5b36a489608 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:35.136697 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c741e2c-c2ce-405e-85b1-7b648cc8b3d5 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:35.138475 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132244af-bd57-4bca-b675-fd870c3095e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:35.139505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132244af-bd57-4bca-b675-fd870c3095e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:35.139793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132244af-bd57-4bca-b675-fd870c3095e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:35.140065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132244af-bd57-4bca-b675-fd870c3095e1 admin admin] PATH_INFO: `/v3/projects/c03b717751ed4cd9ba8189526da46a70/users/c45f1e6b81a441a893a665ce703aea29/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:35.142556 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee9530-5841-435a-bf1d-aa0345af825a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:35.143456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee9530-5841-435a-bf1d-aa0345af825a service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:35.144181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee9530-5841-435a-bf1d-aa0345af825a service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:35.144725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee9530-5841-435a-bf1d-aa0345af825a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:35.156995 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee9530-5841-435a-bf1d-aa0345af825a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:35.158721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee9530-5841-435a-bf1d-aa0345af825a service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:35.176599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1293/2586] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:35.178097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132244af-bd57-4bca-b675-fd870c3095e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c03b717751ed4cd9ba8189526da46a70, user_id=c45f1e6b81a441a893a665ce703aea29, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:35.180885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132244af-bd57-4bca-b675-fd870c3095e1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:35.198683 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1294/2587] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:35 2022] PUT /identity/v3/projects/c03b717751ed4cd9ba8189526da46a70/users/c45f1e6b81a441a893a665ce703aea29/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:35.202977 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee9530-5841-435a-bf1d-aa0345af825a service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:35.222793 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee9530-5841-435a-bf1d-aa0345af825a service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:35.272089 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:35.311880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d57280-4a62-46b7-a721-f5be7c8fd0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:35.312759 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d57280-4a62-46b7-a721-f5be7c8fd0bc admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:35.313062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d57280-4a62-46b7-a721-f5be7c8fd0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:35.313335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d57280-4a62-46b7-a721-f5be7c8fd0bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:35.314083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d57280-4a62-46b7-a721-f5be7c8fd0bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:35.315976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d57280-4a62-46b7-a721-f5be7c8fd0bc admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:35.327289 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1294/2588] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:35.331186 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132244af-bd57-4bca-b675-fd870c3095e1 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:35.352707 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132244af-bd57-4bca-b675-fd870c3095e1 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:35.365563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bee7d4-0e94-408e-9d3b-c4050d2aab72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:35.366476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bee7d4-0e94-408e-9d3b-c4050d2aab72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:35.366757 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bee7d4-0e94-408e-9d3b-c4050d2aab72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:35.367031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bee7d4-0e94-408e-9d3b-c4050d2aab72 admin admin] PATH_INFO: `/v3/projects/c03b717751ed4cd9ba8189526da46a70/users/c45f1e6b81a441a893a665ce703aea29/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:35.369293 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bee7d4-0e94-408e-9d3b-c4050d2aab72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c03b717751ed4cd9ba8189526da46a70, user_id=c45f1e6b81a441a893a665ce703aea29, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:35.372016 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bee7d4-0e94-408e-9d3b-c4050d2aab72 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:35.384072 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1295/2589] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:35 2022] PUT /identity/v3/projects/c03b717751ed4cd9ba8189526da46a70/users/c45f1e6b81a441a893a665ce703aea29/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:30:35.394807 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d9c27-429e-4dd1-8ed7-47be6ba381db None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:35.395097 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d9c27-429e-4dd1-8ed7-47be6ba381db None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:35.395387 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d9c27-429e-4dd1-8ed7-47be6ba381db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:35.425513 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-793d9c27-429e-4dd1-8ed7-47be6ba381db None None] MFA Rules not processed for user `c45f1e6b81a441a893a665ce703aea29`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:35.426739 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-793d9c27-429e-4dd1-8ed7-47be6ba381db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:35.515412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1295/2590] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:35.525268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bee7d4-0e94-408e-9d3b-c4050d2aab72 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:35.527134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-28bee7d4-0e94-408e-9d3b-c4050d2aab72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:35.538714 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d9c27-429e-4dd1-8ed7-47be6ba381db None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:35.541070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-793d9c27-429e-4dd1-8ed7-47be6ba381db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:35.627393 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bee7d4-0e94-408e-9d3b-c4050d2aab72 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:35.638083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d9c27-429e-4dd1-8ed7-47be6ba381db None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:35.638274 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:35.639021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:35.639307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:35.639605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1 admin admin] PATH_INFO: `/v3/projects/f4003caa05e1441591a307083ef56947` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:35.640840 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4003caa05e1441591a307083ef56947)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:35.642925 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:35.643405 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f88fc-5f2d-484c-89d4-0fb45109d881 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:35.643935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f88fc-5f2d-484c-89d4-0fb45109d881 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:35.644212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f88fc-5f2d-484c-89d4-0fb45109d881 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:35.644750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f88fc-5f2d-484c-89d4-0fb45109d881 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:35.655582 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f88fc-5f2d-484c-89d4-0fb45109d881 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:35.657186 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f88fc-5f2d-484c-89d4-0fb45109d881 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:35.668372 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1296/2591] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:30:35.702359 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4003caa05e1441591a307083ef56947', 'request_id': 'req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.709528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4003caa05e1441591a307083ef56947', 'request_id': 'req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.713849 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f88fc-5f2d-484c-89d4-0fb45109d881 service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:35.716463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-499f88fc-5f2d-484c-89d4-0fb45109d881 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:35.721411 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4003caa05e1441591a307083ef56947', 'request_id': 'req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.730680 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f88fc-5f2d-484c-89d4-0fb45109d881 service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:35.738130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1 admin admin] The token cache is being invalidate because project f4003caa05e1441591a307083ef56947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:35.738425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.739192 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.740921 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1296/2592] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:35 2022] DELETE /identity/v3/projects/f4003caa05e1441591a307083ef56947 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:35.744809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:35.746240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:35.760549 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:35.790489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afa257-42f8-41f4-a293-4c4fe139637f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:35.791188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afa257-42f8-41f4-a293-4c4fe139637f service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:35.791542 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afa257-42f8-41f4-a293-4c4fe139637f service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:35.791909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afa257-42f8-41f4-a293-4c4fe139637f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:35.793939 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d6afa257-42f8-41f4-a293-4c4fe139637f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:35.805865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:35.824196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e426a3-1ee8-4bb8-ba58-7af5f87663b1 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:35.832121 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bcda8-75fd-4867-87c0-7fb566b6cf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:35.833081 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bcda8-75fd-4867-87c0-7fb566b6cf7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:35.833530 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bcda8-75fd-4867-87c0-7fb566b6cf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:35.833813 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bcda8-75fd-4867-87c0-7fb566b6cf7b admin admin] PATH_INFO: `/v3/users/98f3dfecc88e48c1a2f6742a6ebf7163` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:35.835188 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bcda8-75fd-4867-87c0-7fb566b6cf7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=98f3dfecc88e48c1a2f6742a6ebf7163)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:35.837277 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bcda8-75fd-4867-87c0-7fb566b6cf7b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:35.869785 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afa257-42f8-41f4-a293-4c4fe139637f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:35.871349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afa257-42f8-41f4-a293-4c4fe139637f service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:35.881996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1297/2593] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:30:35.894213 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-775bcda8-75fd-4867-87c0-7fb566b6cf7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98f3dfecc88e48c1a2f6742a6ebf7163'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.903844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-775bcda8-75fd-4867-87c0-7fb566b6cf7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98f3dfecc88e48c1a2f6742a6ebf7163'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.905419 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-775bcda8-75fd-4867-87c0-7fb566b6cf7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98f3dfecc88e48c1a2f6742a6ebf7163'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.919132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-775bcda8-75fd-4867-87c0-7fb566b6cf7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98f3dfecc88e48c1a2f6742a6ebf7163'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.934122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-775bcda8-75fd-4867-87c0-7fb566b6cf7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98f3dfecc88e48c1a2f6742a6ebf7163'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.940821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1297/2594] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:35 2022] DELETE /identity/v3/users/98f3dfecc88e48c1a2f6742a6ebf7163 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:30:36.284098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afa257-42f8-41f4-a293-4c4fe139637f service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:36.301756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afa257-42f8-41f4-a293-4c4fe139637f service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:36.399336 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fc36d-6228-4765-843a-2ea322b1fa86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:36.400245 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fc36d-6228-4765-843a-2ea322b1fa86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:36.400555 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fc36d-6228-4765-843a-2ea322b1fa86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:36.400833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fc36d-6228-4765-843a-2ea322b1fa86 admin admin] PATH_INFO: `/v3/projects/be2407582e674aaaadb75f9c32957ab0` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:36.402051 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fc36d-6228-4765-843a-2ea322b1fa86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be2407582e674aaaadb75f9c32957ab0)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:36.404259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fc36d-6228-4765-843a-2ea322b1fa86 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:36.454275 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-336fc36d-6228-4765-843a-2ea322b1fa86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be2407582e674aaaadb75f9c32957ab0', 'request_id': 'req-336fc36d-6228-4765-843a-2ea322b1fa86'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.462493 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-336fc36d-6228-4765-843a-2ea322b1fa86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be2407582e674aaaadb75f9c32957ab0', 'request_id': 'req-336fc36d-6228-4765-843a-2ea322b1fa86'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.468700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-336fc36d-6228-4765-843a-2ea322b1fa86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be2407582e674aaaadb75f9c32957ab0', 'request_id': 'req-336fc36d-6228-4765-843a-2ea322b1fa86'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.481573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-336fc36d-6228-4765-843a-2ea322b1fa86 admin admin] The token cache is being invalidate because project be2407582e674aaaadb75f9c32957ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:36.481955 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-336fc36d-6228-4765-843a-2ea322b1fa86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.482703 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-336fc36d-6228-4765-843a-2ea322b1fa86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.484361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1298/2595] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:36 2022] DELETE /identity/v3/projects/be2407582e674aaaadb75f9c32957ab0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:36.488407 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bcda8-75fd-4867-87c0-7fb566b6cf7b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:36.489932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-775bcda8-75fd-4867-87c0-7fb566b6cf7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:36.504072 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bcda8-75fd-4867-87c0-7fb566b6cf7b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:36.562216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da208292-914b-4a63-9cfe-27be2d2bbc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:36.563019 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da208292-914b-4a63-9cfe-27be2d2bbc8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:36.563299 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da208292-914b-4a63-9cfe-27be2d2bbc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:36.563570 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da208292-914b-4a63-9cfe-27be2d2bbc8c admin admin] PATH_INFO: `/v3/users/3fd9a1da5d624ff683f145f8dea7235c` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:36.564911 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da208292-914b-4a63-9cfe-27be2d2bbc8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fd9a1da5d624ff683f145f8dea7235c)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:36.567066 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da208292-914b-4a63-9cfe-27be2d2bbc8c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:36.606738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:36.618374 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-da208292-914b-4a63-9cfe-27be2d2bbc8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fd9a1da5d624ff683f145f8dea7235c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.628242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-da208292-914b-4a63-9cfe-27be2d2bbc8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fd9a1da5d624ff683f145f8dea7235c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.628920 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-da208292-914b-4a63-9cfe-27be2d2bbc8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fd9a1da5d624ff683f145f8dea7235c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.636262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-da208292-914b-4a63-9cfe-27be2d2bbc8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fd9a1da5d624ff683f145f8dea7235c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.649325 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-da208292-914b-4a63-9cfe-27be2d2bbc8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fd9a1da5d624ff683f145f8dea7235c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.655539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1298/2596] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:36 2022] DELETE /identity/v3/users/3fd9a1da5d624ff683f145f8dea7235c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:36.950181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fc36d-6228-4765-843a-2ea322b1fa86 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:36.951728 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-336fc36d-6228-4765-843a-2ea322b1fa86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:36.967368 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fc36d-6228-4765-843a-2ea322b1fa86 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.016121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:37.039641 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18ba9a-1896-49a9-acdc-25cc8c693e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.040660 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18ba9a-1896-49a9-acdc-25cc8c693e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.040948 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18ba9a-1896-49a9-acdc-25cc8c693e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.041222 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18ba9a-1896-49a9-acdc-25cc8c693e67 admin admin] PATH_INFO: `/v3/projects/d99168901cf84cc2ba2662b05f70c8fd` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.042420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a18ba9a-1896-49a9-acdc-25cc8c693e67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d99168901cf84cc2ba2662b05f70c8fd)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.045056 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a18ba9a-1896-49a9-acdc-25cc8c693e67 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.095476 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9a18ba9a-1896-49a9-acdc-25cc8c693e67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd99168901cf84cc2ba2662b05f70c8fd', 'request_id': 'req-9a18ba9a-1896-49a9-acdc-25cc8c693e67'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:37.103377 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9a18ba9a-1896-49a9-acdc-25cc8c693e67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd99168901cf84cc2ba2662b05f70c8fd', 'request_id': 'req-9a18ba9a-1896-49a9-acdc-25cc8c693e67'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:37.111102 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9a18ba9a-1896-49a9-acdc-25cc8c693e67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd99168901cf84cc2ba2662b05f70c8fd', 'request_id': 'req-9a18ba9a-1896-49a9-acdc-25cc8c693e67'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:37.122686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9a18ba9a-1896-49a9-acdc-25cc8c693e67 admin admin] The token cache is being invalidate because project d99168901cf84cc2ba2662b05f70c8fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:37.122986 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9a18ba9a-1896-49a9-acdc-25cc8c693e67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:37.123641 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-9a18ba9a-1896-49a9-acdc-25cc8c693e67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:37.125217 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1299/2597] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:36 2022] DELETE /identity/v3/projects/d99168901cf84cc2ba2662b05f70c8fd => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:37.131492 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3af3f9-52fd-48a5-99cc-0acec0a75c8b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.131786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3af3f9-52fd-48a5-99cc-0acec0a75c8b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.132097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3af3f9-52fd-48a5-99cc-0acec0a75c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.150377 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ab3af3f9-52fd-48a5-99cc-0acec0a75c8b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:37.151592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ab3af3f9-52fd-48a5-99cc-0acec0a75c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:37.214687 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1299/2598] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:37.223813 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff766c61-f2dd-4446-9a26-8eb68ddc764f None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.224101 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff766c61-f2dd-4446-9a26-8eb68ddc764f None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.224466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff766c61-f2dd-4446-9a26-8eb68ddc764f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.243477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-ff766c61-f2dd-4446-9a26-8eb68ddc764f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:37.244801 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ff766c61-f2dd-4446-9a26-8eb68ddc764f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:37.253245 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1300/2599] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:37.257496 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3af3f9-52fd-48a5-99cc-0acec0a75c8b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.269640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3af3f9-52fd-48a5-99cc-0acec0a75c8b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.274728 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa1385-8ff4-4ed8-8a4b-3f7b962aa845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.275388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa1385-8ff4-4ed8-8a4b-3f7b962aa845 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.275669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa1385-8ff4-4ed8-8a4b-3f7b962aa845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.275942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa1385-8ff4-4ed8-8a4b-3f7b962aa845 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.277377 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa1385-8ff4-4ed8-8a4b-3f7b962aa845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:30:37.277686 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa1385-8ff4-4ed8-8a4b-3f7b962aa845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.279906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa1385-8ff4-4ed8-8a4b-3f7b962aa845 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.300331 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1300/2600] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:30:37.302965 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff766c61-f2dd-4446-9a26-8eb68ddc764f None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.313611 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff766c61-f2dd-4446-9a26-8eb68ddc764f None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.319143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52850e60-7579-47b8-ae0b-0bf0d482c030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.319824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52850e60-7579-47b8-ae0b-0bf0d482c030 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.320110 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52850e60-7579-47b8-ae0b-0bf0d482c030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.320540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52850e60-7579-47b8-ae0b-0bf0d482c030 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.321261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52850e60-7579-47b8-ae0b-0bf0d482c030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.323145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52850e60-7579-47b8-ae0b-0bf0d482c030 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.356610 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa1385-8ff4-4ed8-8a4b-3f7b962aa845 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.360442 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1301/2601] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:37 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:37.368437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa1385-8ff4-4ed8-8a4b-3f7b962aa845 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.437352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952bc68-e264-43df-b161-0a01a569cc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.438673 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952bc68-e264-43df-b161-0a01a569cc31 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.438965 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952bc68-e264-43df-b161-0a01a569cc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.439247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952bc68-e264-43df-b161-0a01a569cc31 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.439959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952bc68-e264-43df-b161-0a01a569cc31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.442198 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952bc68-e264-43df-b161-0a01a569cc31 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.463483 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1301/2602] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:37 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:30:37.468524 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52850e60-7579-47b8-ae0b-0bf0d482c030 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.479641 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52850e60-7579-47b8-ae0b-0bf0d482c030 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.484740 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83821a-2ea3-4ed6-8acd-58602769d08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.485407 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83821a-2ea3-4ed6-8acd-58602769d08f admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.485690 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83821a-2ea3-4ed6-8acd-58602769d08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.485996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83821a-2ea3-4ed6-8acd-58602769d08f admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.486725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83821a-2ea3-4ed6-8acd-58602769d08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.488335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83821a-2ea3-4ed6-8acd-58602769d08f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.504248 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1302/2603] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:37.505808 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952bc68-e264-43df-b161-0a01a569cc31 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.520060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952bc68-e264-43df-b161-0a01a569cc31 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.528902 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b26e19-0a91-40e8-b4ab-addded7aadf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.529985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b26e19-0a91-40e8-b4ab-addded7aadf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.530276 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b26e19-0a91-40e8-b4ab-addded7aadf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.530555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b26e19-0a91-40e8-b4ab-addded7aadf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.531297 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b26e19-0a91-40e8-b4ab-addded7aadf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.532980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b26e19-0a91-40e8-b4ab-addded7aadf9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.544333 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1302/2604] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:37.548684 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83821a-2ea3-4ed6-8acd-58602769d08f admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.560261 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83821a-2ea3-4ed6-8acd-58602769d08f admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.567858 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d1e58-78b7-421f-a8db-af47b747c9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.568908 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d1e58-78b7-421f-a8db-af47b747c9f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.569216 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d1e58-78b7-421f-a8db-af47b747c9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.569510 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d1e58-78b7-421f-a8db-af47b747c9f7 admin admin] PATH_INFO: `/v3/projects/8b8ab2928f6c498880400f5dd3f5323b/users/ea873f126def44f8a904c55e79b53a75/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.592339 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d1e58-78b7-421f-a8db-af47b747c9f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b8ab2928f6c498880400f5dd3f5323b, user_id=ea873f126def44f8a904c55e79b53a75, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.594809 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d1e58-78b7-421f-a8db-af47b747c9f7 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.610898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1303/2605] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:37 2022] PUT /identity/v3/projects/8b8ab2928f6c498880400f5dd3f5323b/users/ea873f126def44f8a904c55e79b53a75/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:37.623253 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186bba3-ab60-4067-8678-6d161488d57d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.623594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186bba3-ab60-4067-8678-6d161488d57d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.623827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186bba3-ab60-4067-8678-6d161488d57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.657869 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-6186bba3-ab60-4067-8678-6d161488d57d None None] MFA Rules not processed for user `ea873f126def44f8a904c55e79b53a75`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:37.659125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6186bba3-ab60-4067-8678-6d161488d57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:37.665213 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:37.735765 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1303/2606] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:37.740472 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d1e58-78b7-421f-a8db-af47b747c9f7 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.814091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d1e58-78b7-421f-a8db-af47b747c9f7 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.819357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc36f3c-74ab-4bc1-a259-28c0842001c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.820032 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc36f3c-74ab-4bc1-a259-28c0842001c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.820349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc36f3c-74ab-4bc1-a259-28c0842001c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.821410 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc36f3c-74ab-4bc1-a259-28c0842001c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.821748 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc36f3c-74ab-4bc1-a259-28c0842001c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.823636 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc36f3c-74ab-4bc1-a259-28c0842001c1 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.839621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:37.856077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1304/2607] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:37 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:37.860556 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186bba3-ab60-4067-8678-6d161488d57d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.873193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186bba3-ab60-4067-8678-6d161488d57d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.933653 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06818b88-e54f-4b37-9eb5-557ae6bf94fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.934933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06818b88-e54f-4b37-9eb5-557ae6bf94fd admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.935164 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06818b88-e54f-4b37-9eb5-557ae6bf94fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.935443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06818b88-e54f-4b37-9eb5-557ae6bf94fd admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.936194 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06818b88-e54f-4b37-9eb5-557ae6bf94fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.938785 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06818b88-e54f-4b37-9eb5-557ae6bf94fd admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.964337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1304/2608] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:37 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:37.967068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc36f3c-74ab-4bc1-a259-28c0842001c1 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.980661 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc36f3c-74ab-4bc1-a259-28c0842001c1 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.985965 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b035fb-4534-45e4-bb5f-47064d775a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.986624 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b035fb-4534-45e4-bb5f-47064d775a92 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.986920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b035fb-4534-45e4-bb5f-47064d775a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.987195 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b035fb-4534-45e4-bb5f-47064d775a92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.987926 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b035fb-4534-45e4-bb5f-47064d775a92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.989543 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b035fb-4534-45e4-bb5f-47064d775a92 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:38.001365 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1305/2609] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:38.005632 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06818b88-e54f-4b37-9eb5-557ae6bf94fd admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:38.016150 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06818b88-e54f-4b37-9eb5-557ae6bf94fd admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:38.021297 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398bf795-99fe-4951-bdbf-9eff59bb9bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:38.022422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bf795-99fe-4951-bdbf-9eff59bb9bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:38.022772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bf795-99fe-4951-bdbf-9eff59bb9bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:38.023005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bf795-99fe-4951-bdbf-9eff59bb9bbe admin admin] PATH_INFO: `/v3/projects/34d54d74b14047f7bad701daea4cd5bb/users/3a7cd5d73dde4e71be984d515be21f9b/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:38.045555 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398bf795-99fe-4951-bdbf-9eff59bb9bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=34d54d74b14047f7bad701daea4cd5bb, user_id=3a7cd5d73dde4e71be984d515be21f9b, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:38.048763 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398bf795-99fe-4951-bdbf-9eff59bb9bbe admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:38.064930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1305/2610] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:38 2022] PUT /identity/v3/projects/34d54d74b14047f7bad701daea4cd5bb/users/3a7cd5d73dde4e71be984d515be21f9b/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:30:38.070342 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b035fb-4534-45e4-bb5f-47064d775a92 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:38.087606 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b035fb-4534-45e4-bb5f-47064d775a92 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:38.154435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6a74b-c05f-4bfe-8c63-316844b15807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:38.155287 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6a74b-c05f-4bfe-8c63-316844b15807 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:38.155588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6a74b-c05f-4bfe-8c63-316844b15807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:38.155913 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6a74b-c05f-4bfe-8c63-316844b15807 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:38.156630 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6a74b-c05f-4bfe-8c63-316844b15807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:38.158455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6a74b-c05f-4bfe-8c63-316844b15807 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:38.173388 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1306/2611] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:38.177839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398bf795-99fe-4951-bdbf-9eff59bb9bbe admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:38.190922 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398bf795-99fe-4951-bdbf-9eff59bb9bbe admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:38.196620 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311789be-91c8-438f-a243-bc2f0bc24ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:38.198089 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311789be-91c8-438f-a243-bc2f0bc24ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:38.198344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311789be-91c8-438f-a243-bc2f0bc24ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:38.198618 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311789be-91c8-438f-a243-bc2f0bc24ea3 admin admin] PATH_INFO: `/v3/projects/34d54d74b14047f7bad701daea4cd5bb/users/3a7cd5d73dde4e71be984d515be21f9b/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:38.200745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311789be-91c8-438f-a243-bc2f0bc24ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34d54d74b14047f7bad701daea4cd5bb, user_id=3a7cd5d73dde4e71be984d515be21f9b, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:38.203639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311789be-91c8-438f-a243-bc2f0bc24ea3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:38.217058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1306/2612] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:38 2022] PUT /identity/v3/projects/34d54d74b14047f7bad701daea4cd5bb/users/3a7cd5d73dde4e71be984d515be21f9b/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:38.230565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f013bd0-77bc-485c-9127-67c1e39fee13 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:38.230848 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f013bd0-77bc-485c-9127-67c1e39fee13 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:38.231669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f013bd0-77bc-485c-9127-67c1e39fee13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:38.259935 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-9f013bd0-77bc-485c-9127-67c1e39fee13 None None] MFA Rules not processed for user `3a7cd5d73dde4e71be984d515be21f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:38.261209 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-9f013bd0-77bc-485c-9127-67c1e39fee13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:38.358402 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1307/2613] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:38.378259 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311789be-91c8-438f-a243-bc2f0bc24ea3 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:38.396157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-311789be-91c8-438f-a243-bc2f0bc24ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:38.461696 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dac158-b928-4ed9-b836-3db8c7617912 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:38.462155 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dac158-b928-4ed9-b836-3db8c7617912 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:38.462531 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dac158-b928-4ed9-b836-3db8c7617912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:38.485843 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-75dac158-b928-4ed9-b836-3db8c7617912 None None] MFA Rules not processed for user `4a4f4d585adf40b880be44c29ecd53cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:38.487198 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-75dac158-b928-4ed9-b836-3db8c7617912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:38.510004 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311789be-91c8-438f-a243-bc2f0bc24ea3 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:38.510004 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bd85e-fa1e-43a5-ac03-4b9a41b241e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:38.510004 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bd85e-fa1e-43a5-ac03-4b9a41b241e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:38.510004 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bd85e-fa1e-43a5-ac03-4b9a41b241e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:38.510401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bd85e-fa1e-43a5-ac03-4b9a41b241e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:38.520897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17bd85e-fa1e-43a5-ac03-4b9a41b241e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:38.522534 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17bd85e-fa1e-43a5-ac03-4b9a41b241e2 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:38.538225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1307/2614] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:38.577354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1308/2615] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:38.899866 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bd85e-fa1e-43a5-ac03-4b9a41b241e2 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:38.905377 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:39.059873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bd85e-fa1e-43a5-ac03-4b9a41b241e2 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:39.067601 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8443c-fb5b-4622-a2bb-06fbe210b3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:39.068982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8443c-fb5b-4622-a2bb-06fbe210b3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:39.069418 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8443c-fb5b-4622-a2bb-06fbe210b3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:39.069846 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8443c-fb5b-4622-a2bb-06fbe210b3e9 admin admin] PATH_INFO: `/v3/users/ea873f126def44f8a904c55e79b53a75` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:39.072048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8443c-fb5b-4622-a2bb-06fbe210b3e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea873f126def44f8a904c55e79b53a75)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:39.075686 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8443c-fb5b-4622-a2bb-06fbe210b3e9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:39.080480 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dac158-b928-4ed9-b836-3db8c7617912 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:39.082858 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-75dac158-b928-4ed9-b836-3db8c7617912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:39.109945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:39.169634 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a9e8443c-fb5b-4622-a2bb-06fbe210b3e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea873f126def44f8a904c55e79b53a75'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:39.186228 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a9e8443c-fb5b-4622-a2bb-06fbe210b3e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea873f126def44f8a904c55e79b53a75'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:39.189096 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a9e8443c-fb5b-4622-a2bb-06fbe210b3e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea873f126def44f8a904c55e79b53a75'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:39.198856 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a9e8443c-fb5b-4622-a2bb-06fbe210b3e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea873f126def44f8a904c55e79b53a75'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:39.207663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dac158-b928-4ed9-b836-3db8c7617912 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:39.211834 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8475fb-78c6-47bc-933d-8c8425727207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:39.212518 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8475fb-78c6-47bc-933d-8c8425727207 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:39.212924 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8475fb-78c6-47bc-933d-8c8425727207 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:39.213202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8475fb-78c6-47bc-933d-8c8425727207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:39.215123 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-2e8475fb-78c6-47bc-933d-8c8425727207 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:39.218007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a9e8443c-fb5b-4622-a2bb-06fbe210b3e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea873f126def44f8a904c55e79b53a75'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:39.226327 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8475fb-78c6-47bc-933d-8c8425727207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:39.226495 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1308/2616] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:38 2022] DELETE /identity/v3/users/ea873f126def44f8a904c55e79b53a75 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:39.229864 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8475fb-78c6-47bc-933d-8c8425727207 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:39.231296 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f9413-60ba-4cfd-a92d-28e050a28ea0 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:39.231625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f9413-60ba-4cfd-a92d-28e050a28ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:39.232159 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f9413-60ba-4cfd-a92d-28e050a28ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:39.245822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1309/2617] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:30:39.253430 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a21f9413-60ba-4cfd-a92d-28e050a28ea0 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:39.254639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a21f9413-60ba-4cfd-a92d-28e050a28ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:39.350235 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1309/2618] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:39.363049 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8475fb-78c6-47bc-933d-8c8425727207 service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:39.368475 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-2e8475fb-78c6-47bc-933d-8c8425727207 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:39.389109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8475fb-78c6-47bc-933d-8c8425727207 service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:39.470153 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe69d28-aa0a-44f8-aad9-050b181dc3cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:39.470911 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe69d28-aa0a-44f8-aad9-050b181dc3cc service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:39.471196 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe69d28-aa0a-44f8-aad9-050b181dc3cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:39.471478 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe69d28-aa0a-44f8-aad9-050b181dc3cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:39.482228 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe69d28-aa0a-44f8-aad9-050b181dc3cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:39.483664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe69d28-aa0a-44f8-aad9-050b181dc3cc service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:39.500566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1310/2619] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:39.854025 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f9413-60ba-4cfd-a92d-28e050a28ea0 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:39.855704 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a21f9413-60ba-4cfd-a92d-28e050a28ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:39.872954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f9413-60ba-4cfd-a92d-28e050a28ea0 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:39.884485 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c34c8-2665-4998-9299-be14b0a22095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:39.886336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c34c8-2665-4998-9299-be14b0a22095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:39.886881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c34c8-2665-4998-9299-be14b0a22095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:39.887175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c34c8-2665-4998-9299-be14b0a22095 admin admin] PATH_INFO: `/v3/projects/8b8ab2928f6c498880400f5dd3f5323b` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:39.889928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c34c8-2665-4998-9299-be14b0a22095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b8ab2928f6c498880400f5dd3f5323b)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:39.891947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c34c8-2665-4998-9299-be14b0a22095 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:39.913337 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:39.953300 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3f6c34c8-2665-4998-9299-be14b0a22095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b8ab2928f6c498880400f5dd3f5323b', 'request_id': 'req-3f6c34c8-2665-4998-9299-be14b0a22095'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:39.960432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3f6c34c8-2665-4998-9299-be14b0a22095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b8ab2928f6c498880400f5dd3f5323b', 'request_id': 'req-3f6c34c8-2665-4998-9299-be14b0a22095'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:39.969253 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3f6c34c8-2665-4998-9299-be14b0a22095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b8ab2928f6c498880400f5dd3f5323b', 'request_id': 'req-3f6c34c8-2665-4998-9299-be14b0a22095'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:39.983749 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3f6c34c8-2665-4998-9299-be14b0a22095 admin admin] The token cache is being invalidate because project 8b8ab2928f6c498880400f5dd3f5323b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:39.984049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3f6c34c8-2665-4998-9299-be14b0a22095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:39.984991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3f6c34c8-2665-4998-9299-be14b0a22095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:39.986312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1310/2620] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:39 2022] DELETE /identity/v3/projects/8b8ab2928f6c498880400f5dd3f5323b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:39.990298 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe69d28-aa0a-44f8-aad9-050b181dc3cc service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:39.999886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe69d28-aa0a-44f8-aad9-050b181dc3cc service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:40.067052 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738f331-5f8e-448d-b62b-b8ff9bb0ebba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:40.067940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738f331-5f8e-448d-b62b-b8ff9bb0ebba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:40.068222 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738f331-5f8e-448d-b62b-b8ff9bb0ebba admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:40.068539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738f331-5f8e-448d-b62b-b8ff9bb0ebba admin admin] PATH_INFO: `/v3/users/3a7cd5d73dde4e71be984d515be21f9b` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:40.069732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738f331-5f8e-448d-b62b-b8ff9bb0ebba admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a7cd5d73dde4e71be984d515be21f9b)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:40.071637 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738f331-5f8e-448d-b62b-b8ff9bb0ebba admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:40.123647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-0738f331-5f8e-448d-b62b-b8ff9bb0ebba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a7cd5d73dde4e71be984d515be21f9b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:40.132409 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-0738f331-5f8e-448d-b62b-b8ff9bb0ebba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a7cd5d73dde4e71be984d515be21f9b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:40.135375 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:40.152458 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-0738f331-5f8e-448d-b62b-b8ff9bb0ebba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a7cd5d73dde4e71be984d515be21f9b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:40.160978 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-0738f331-5f8e-448d-b62b-b8ff9bb0ebba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a7cd5d73dde4e71be984d515be21f9b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:40.161714 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-0738f331-5f8e-448d-b62b-b8ff9bb0ebba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a7cd5d73dde4e71be984d515be21f9b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:40.168663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1311/2621] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:39 2022] DELETE /identity/v3/users/3a7cd5d73dde4e71be984d515be21f9b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:40.368541 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c34c8-2665-4998-9299-be14b0a22095 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:40.370420 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3f6c34c8-2665-4998-9299-be14b0a22095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:40.386893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c34c8-2665-4998-9299-be14b0a22095 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:40.468108 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9ef92-3c42-4bf8-a73d-217d0643f42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:40.469197 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9ef92-3c42-4bf8-a73d-217d0643f42b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:40.469489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9ef92-3c42-4bf8-a73d-217d0643f42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:40.469767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9ef92-3c42-4bf8-a73d-217d0643f42b admin admin] PATH_INFO: `/v3/users/81d6d182cdd742cbb34707da109c985f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:40.471402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9ef92-3c42-4bf8-a73d-217d0643f42b admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d6d182cdd742cbb34707da109c985f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:40.473102 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9ef92-3c42-4bf8-a73d-217d0643f42b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:40.536970 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-48b9ef92-3c42-4bf8-a73d-217d0643f42b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d6d182cdd742cbb34707da109c985f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:40.553060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-48b9ef92-3c42-4bf8-a73d-217d0643f42b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d6d182cdd742cbb34707da109c985f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:40.553752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-48b9ef92-3c42-4bf8-a73d-217d0643f42b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d6d182cdd742cbb34707da109c985f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:40.562642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-48b9ef92-3c42-4bf8-a73d-217d0643f42b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d6d182cdd742cbb34707da109c985f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:40.575641 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-48b9ef92-3c42-4bf8-a73d-217d0643f42b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d6d182cdd742cbb34707da109c985f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:40.581674 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1311/2622] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:40 2022] DELETE /identity/v3/users/81d6d182cdd742cbb34707da109c985f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:40.586351 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c95348-252b-4741-9b74-e315f1bd4020 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:40.587143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c95348-252b-4741-9b74-e315f1bd4020 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:40.587143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c95348-252b-4741-9b74-e315f1bd4020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:40.604851 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-13c95348-252b-4741-9b74-e315f1bd4020 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:40.606118 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-13c95348-252b-4741-9b74-e315f1bd4020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:40.620275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9ef92-3c42-4bf8-a73d-217d0643f42b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:40.622374 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-48b9ef92-3c42-4bf8-a73d-217d0643f42b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:40.666347 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:40.681767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1312/2623] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:40.693104 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c95348-252b-4741-9b74-e315f1bd4020 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:40.695465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-13c95348-252b-4741-9b74-e315f1bd4020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:40.725691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9ef92-3c42-4bf8-a73d-217d0643f42b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:40.732807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881e8bf-b44e-4329-99dc-61b825425125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:40.733681 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881e8bf-b44e-4329-99dc-61b825425125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:40.733963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881e8bf-b44e-4329-99dc-61b825425125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:40.734236 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881e8bf-b44e-4329-99dc-61b825425125 admin admin] PATH_INFO: `/v3/projects/34d54d74b14047f7bad701daea4cd5bb` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:40.735443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881e8bf-b44e-4329-99dc-61b825425125 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34d54d74b14047f7bad701daea4cd5bb)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:40.737394 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881e8bf-b44e-4329-99dc-61b825425125 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:40.755687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c95348-252b-4741-9b74-e315f1bd4020 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:40.759316 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0e3c1-fdfc-4716-97c4-2b16604a32c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:40.759970 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0e3c1-fdfc-4716-97c4-2b16604a32c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:40.760257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0e3c1-fdfc-4716-97c4-2b16604a32c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:40.761496 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0e3c1-fdfc-4716-97c4-2b16604a32c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:40.773463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0e3c1-fdfc-4716-97c4-2b16604a32c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:40.775669 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0e3c1-fdfc-4716-97c4-2b16604a32c2 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:40.787812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1313/2624] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:30:40.795429 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f881e8bf-b44e-4329-99dc-61b825425125 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34d54d74b14047f7bad701daea4cd5bb', 'request_id': 'req-f881e8bf-b44e-4329-99dc-61b825425125'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:40.804034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f881e8bf-b44e-4329-99dc-61b825425125 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34d54d74b14047f7bad701daea4cd5bb', 'request_id': 'req-f881e8bf-b44e-4329-99dc-61b825425125'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:40.814312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f881e8bf-b44e-4329-99dc-61b825425125 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34d54d74b14047f7bad701daea4cd5bb', 'request_id': 'req-f881e8bf-b44e-4329-99dc-61b825425125'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:40.832589 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f881e8bf-b44e-4329-99dc-61b825425125 admin admin] The token cache is being invalidate because project 34d54d74b14047f7bad701daea4cd5bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:40.832886 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f881e8bf-b44e-4329-99dc-61b825425125 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:40.833579 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f881e8bf-b44e-4329-99dc-61b825425125 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:40.835777 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1312/2625] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:40 2022] DELETE /identity/v3/projects/34d54d74b14047f7bad701daea4cd5bb => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:40.843350 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadffd5-b948-479a-a7be-a542f525c5bf None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:40.843848 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadffd5-b948-479a-a7be-a542f525c5bf None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:40.844204 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadffd5-b948-479a-a7be-a542f525c5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:40.866467 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-6fadffd5-b948-479a-a7be-a542f525c5bf None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:40.867716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6fadffd5-b948-479a-a7be-a542f525c5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:40.984319 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1314/2626] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:40.994660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d592b6-d0ba-448f-88c1-339b7039288c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:40.996005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d592b6-d0ba-448f-88c1-339b7039288c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:40.997366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d592b6-d0ba-448f-88c1-339b7039288c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:41.020456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-79d592b6-d0ba-448f-88c1-339b7039288c None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:41.021730 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-79d592b6-d0ba-448f-88c1-339b7039288c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:41.031174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1313/2627] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:41.046004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadffd5-b948-479a-a7be-a542f525c5bf None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:41.058656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadffd5-b948-479a-a7be-a542f525c5bf None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:41.072889 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233fc15-7f98-45ad-9ec0-fd339ab42f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:41.073741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233fc15-7f98-45ad-9ec0-fd339ab42f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:41.074068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233fc15-7f98-45ad-9ec0-fd339ab42f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:41.074797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233fc15-7f98-45ad-9ec0-fd339ab42f5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:41.075651 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233fc15-7f98-45ad-9ec0-fd339ab42f5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:30:41.076086 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233fc15-7f98-45ad-9ec0-fd339ab42f5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:41.077956 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233fc15-7f98-45ad-9ec0-fd339ab42f5a admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:41.092266 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1315/2628] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:30:41.097680 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d592b6-d0ba-448f-88c1-339b7039288c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:41.109122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d592b6-d0ba-448f-88c1-339b7039288c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:41.116271 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e36fff-1721-4197-a49e-47a4f33cc443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:41.118558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e36fff-1721-4197-a49e-47a4f33cc443 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:41.120058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e36fff-1721-4197-a49e-47a4f33cc443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:41.120478 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e36fff-1721-4197-a49e-47a4f33cc443 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:41.121746 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e36fff-1721-4197-a49e-47a4f33cc443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:41.126216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e36fff-1721-4197-a49e-47a4f33cc443 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:41.165901 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1314/2629] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:41 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:41.169749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233fc15-7f98-45ad-9ec0-fd339ab42f5a admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:41.183290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:41.189130 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233fc15-7f98-45ad-9ec0-fd339ab42f5a admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:41.269800 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109c4e3-2094-4940-99fb-df7a43c66b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:41.270553 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109c4e3-2094-4940-99fb-df7a43c66b74 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:41.270915 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109c4e3-2094-4940-99fb-df7a43c66b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:41.271277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109c4e3-2094-4940-99fb-df7a43c66b74 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:41.272109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109c4e3-2094-4940-99fb-df7a43c66b74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:41.274324 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109c4e3-2094-4940-99fb-df7a43c66b74 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:41.296423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1316/2630] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:41 2022] POST /identity/v3/users => generated 291 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:41.300675 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e36fff-1721-4197-a49e-47a4f33cc443 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:41.313178 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e36fff-1721-4197-a49e-47a4f33cc443 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:41.320903 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca07775-3ebb-49ab-9a41-82ca7fba237c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:41.321651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca07775-3ebb-49ab-9a41-82ca7fba237c admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:41.321940 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca07775-3ebb-49ab-9a41-82ca7fba237c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:41.322221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca07775-3ebb-49ab-9a41-82ca7fba237c admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:41.323069 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca07775-3ebb-49ab-9a41-82ca7fba237c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:41.324842 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca07775-3ebb-49ab-9a41-82ca7fba237c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:41.342942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1315/2631] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:41.344145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109c4e3-2094-4940-99fb-df7a43c66b74 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:41.360416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109c4e3-2094-4940-99fb-df7a43c66b74 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:41.369965 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3ff7c-b5a9-460d-9cf4-14f2ad011735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:41.371062 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3ff7c-b5a9-460d-9cf4-14f2ad011735 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:41.371952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3ff7c-b5a9-460d-9cf4-14f2ad011735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:41.372366 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3ff7c-b5a9-460d-9cf4-14f2ad011735 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:41.373187 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3ff7c-b5a9-460d-9cf4-14f2ad011735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:41.375385 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3ff7c-b5a9-460d-9cf4-14f2ad011735 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:41.391729 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1317/2632] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:41.395178 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca07775-3ebb-49ab-9a41-82ca7fba237c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:41.406881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca07775-3ebb-49ab-9a41-82ca7fba237c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:41.415266 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40036f51-4798-4f32-b0dc-f715242f1184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:41.416958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40036f51-4798-4f32-b0dc-f715242f1184 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:41.417360 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40036f51-4798-4f32-b0dc-f715242f1184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:41.417747 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40036f51-4798-4f32-b0dc-f715242f1184 admin admin] PATH_INFO: `/v3/projects/7ee2fc4ab6a540cc852d2b3850e6b550/users/44d26a97291e44c78d3a6438d7ab76c5/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:41.445049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40036f51-4798-4f32-b0dc-f715242f1184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ee2fc4ab6a540cc852d2b3850e6b550, user_id=44d26a97291e44c78d3a6438d7ab76c5, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:41.449199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40036f51-4798-4f32-b0dc-f715242f1184 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:41.475581 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1316/2633] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:41 2022] PUT /identity/v3/projects/7ee2fc4ab6a540cc852d2b3850e6b550/users/44d26a97291e44c78d3a6438d7ab76c5/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:41.478000 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c5171-f9f4-4508-8759-add0784700ea None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:41.478329 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c5171-f9f4-4508-8759-add0784700ea None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:41.478614 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c5171-f9f4-4508-8759-add0784700ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:41.534692 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-764c5171-f9f4-4508-8759-add0784700ea None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:41.539312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-764c5171-f9f4-4508-8759-add0784700ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:41.653910 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40036f51-4798-4f32-b0dc-f715242f1184 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:41.655760 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-40036f51-4798-4f32-b0dc-f715242f1184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:41.733733 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1318/2634] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:41.759591 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c5171-f9f4-4508-8759-add0784700ea None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:41.762403 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-764c5171-f9f4-4508-8759-add0784700ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:41.793093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:41.812832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40036f51-4798-4f32-b0dc-f715242f1184 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:41.818904 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094fc5d-ca7b-49d9-a600-613da51f03f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:41.820134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094fc5d-ca7b-49d9-a600-613da51f03f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:41.820524 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094fc5d-ca7b-49d9-a600-613da51f03f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:41.821189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094fc5d-ca7b-49d9-a600-613da51f03f7 admin admin] PATH_INFO: `/v3/projects/68267c8e8240468cb76f7aea8c8f994c` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:41.822297 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8094fc5d-ca7b-49d9-a600-613da51f03f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68267c8e8240468cb76f7aea8c8f994c)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:41.824465 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8094fc5d-ca7b-49d9-a600-613da51f03f7 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:41.839446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c5171-f9f4-4508-8759-add0784700ea None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:41.843641 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835a3af-47b2-4ecc-b88a-90f5289a16da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:41.845414 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835a3af-47b2-4ecc-b88a-90f5289a16da service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:41.845777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835a3af-47b2-4ecc-b88a-90f5289a16da service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:41.846352 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835a3af-47b2-4ecc-b88a-90f5289a16da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:41.866875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835a3af-47b2-4ecc-b88a-90f5289a16da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:41.868567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835a3af-47b2-4ecc-b88a-90f5289a16da service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:41.892183 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1319/2635] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:41.904133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8094fc5d-ca7b-49d9-a600-613da51f03f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68267c8e8240468cb76f7aea8c8f994c', 'request_id': 'req-8094fc5d-ca7b-49d9-a600-613da51f03f7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:41.911894 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8094fc5d-ca7b-49d9-a600-613da51f03f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68267c8e8240468cb76f7aea8c8f994c', 'request_id': 'req-8094fc5d-ca7b-49d9-a600-613da51f03f7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:41.923355 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8094fc5d-ca7b-49d9-a600-613da51f03f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68267c8e8240468cb76f7aea8c8f994c', 'request_id': 'req-8094fc5d-ca7b-49d9-a600-613da51f03f7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:41.944924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8094fc5d-ca7b-49d9-a600-613da51f03f7 admin admin] The token cache is being invalidate because project 68267c8e8240468cb76f7aea8c8f994c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:41.945580 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8094fc5d-ca7b-49d9-a600-613da51f03f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:41.947098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8094fc5d-ca7b-49d9-a600-613da51f03f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:41.952380 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1317/2636] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:41 2022] DELETE /identity/v3/projects/68267c8e8240468cb76f7aea8c8f994c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:41.956612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835a3af-47b2-4ecc-b88a-90f5289a16da service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:41.975057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835a3af-47b2-4ecc-b88a-90f5289a16da service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:42.021752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:42.068956 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781b2ba-5b97-4b5f-8e1c-93fe2ed71109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:42.070344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781b2ba-5b97-4b5f-8e1c-93fe2ed71109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:42.070727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781b2ba-5b97-4b5f-8e1c-93fe2ed71109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:42.071106 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781b2ba-5b97-4b5f-8e1c-93fe2ed71109 admin admin] PATH_INFO: `/v3/users/5ece30e2bb834d9b85f71b519d26af41` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:42.072581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2781b2ba-5b97-4b5f-8e1c-93fe2ed71109 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ece30e2bb834d9b85f71b519d26af41)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:42.075034 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2781b2ba-5b97-4b5f-8e1c-93fe2ed71109 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:42.132246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-2781b2ba-5b97-4b5f-8e1c-93fe2ed71109 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ece30e2bb834d9b85f71b519d26af41'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.141154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-2781b2ba-5b97-4b5f-8e1c-93fe2ed71109 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ece30e2bb834d9b85f71b519d26af41'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.164502 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-2781b2ba-5b97-4b5f-8e1c-93fe2ed71109 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ece30e2bb834d9b85f71b519d26af41'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.182620 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-2781b2ba-5b97-4b5f-8e1c-93fe2ed71109 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ece30e2bb834d9b85f71b519d26af41'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.184423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-2781b2ba-5b97-4b5f-8e1c-93fe2ed71109 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ece30e2bb834d9b85f71b519d26af41'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.209899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1320/2637] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:41 2022] DELETE /identity/v3/users/5ece30e2bb834d9b85f71b519d26af41 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:43.030211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094fc5d-ca7b-49d9-a600-613da51f03f7 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:43.031816 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8094fc5d-ca7b-49d9-a600-613da51f03f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:43.059149 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094fc5d-ca7b-49d9-a600-613da51f03f7 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:43.182110 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bfef8-1436-4954-9a78-694ee3b093d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:43.183336 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bfef8-1436-4954-9a78-694ee3b093d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:43.183705 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bfef8-1436-4954-9a78-694ee3b093d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:43.184149 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bfef8-1436-4954-9a78-694ee3b093d3 admin admin] PATH_INFO: `/v3/projects/ac6a55f67d174d6bb9974062231afb3c` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:43.186031 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624bfef8-1436-4954-9a78-694ee3b093d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac6a55f67d174d6bb9974062231afb3c)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:43.188625 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624bfef8-1436-4954-9a78-694ee3b093d3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:43.259849 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-624bfef8-1436-4954-9a78-694ee3b093d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac6a55f67d174d6bb9974062231afb3c', 'request_id': 'req-624bfef8-1436-4954-9a78-694ee3b093d3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:43.268058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-624bfef8-1436-4954-9a78-694ee3b093d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac6a55f67d174d6bb9974062231afb3c', 'request_id': 'req-624bfef8-1436-4954-9a78-694ee3b093d3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:43.278345 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-624bfef8-1436-4954-9a78-694ee3b093d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac6a55f67d174d6bb9974062231afb3c', 'request_id': 'req-624bfef8-1436-4954-9a78-694ee3b093d3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:43.296174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-624bfef8-1436-4954-9a78-694ee3b093d3 admin admin] The token cache is being invalidate because project ac6a55f67d174d6bb9974062231afb3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:43.297115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-624bfef8-1436-4954-9a78-694ee3b093d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:43.297491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-624bfef8-1436-4954-9a78-694ee3b093d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:43.300067 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1318/2638] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:43 2022] DELETE /identity/v3/projects/ac6a55f67d174d6bb9974062231afb3c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:43.304382 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781b2ba-5b97-4b5f-8e1c-93fe2ed71109 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:43.306070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-2781b2ba-5b97-4b5f-8e1c-93fe2ed71109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:43.325674 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781b2ba-5b97-4b5f-8e1c-93fe2ed71109 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:43.452433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68ab44-56a8-4345-b0ec-93ad2c986f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:43.453504 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68ab44-56a8-4345-b0ec-93ad2c986f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:43.454146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68ab44-56a8-4345-b0ec-93ad2c986f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:43.454146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68ab44-56a8-4345-b0ec-93ad2c986f07 admin admin] PATH_INFO: `/v3/users/c45f1e6b81a441a893a665ce703aea29` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:43.455722 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68ab44-56a8-4345-b0ec-93ad2c986f07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c45f1e6b81a441a893a665ce703aea29)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:43.457836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68ab44-56a8-4345-b0ec-93ad2c986f07 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:43.526716 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5e68ab44-56a8-4345-b0ec-93ad2c986f07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c45f1e6b81a441a893a665ce703aea29'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:43.530749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:43.541343 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5e68ab44-56a8-4345-b0ec-93ad2c986f07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c45f1e6b81a441a893a665ce703aea29'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:43.558933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5e68ab44-56a8-4345-b0ec-93ad2c986f07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c45f1e6b81a441a893a665ce703aea29'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:43.576156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5e68ab44-56a8-4345-b0ec-93ad2c986f07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c45f1e6b81a441a893a665ce703aea29'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:43.576970 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5e68ab44-56a8-4345-b0ec-93ad2c986f07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c45f1e6b81a441a893a665ce703aea29'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:43.597872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1321/2639] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:43 2022] DELETE /identity/v3/users/c45f1e6b81a441a893a665ce703aea29 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:30:43.904015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e12800-e7a8-4ddd-9e87-73dedae42395 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:43.904603 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e12800-e7a8-4ddd-9e87-73dedae42395 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:43.905145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e12800-e7a8-4ddd-9e87-73dedae42395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:43.957787 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:43.987041 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-15e12800-e7a8-4ddd-9e87-73dedae42395 None None] MFA Rules not processed for user `44d26a97291e44c78d3a6438d7ab76c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:43.990817 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-15e12800-e7a8-4ddd-9e87-73dedae42395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:44.123779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68ab44-56a8-4345-b0ec-93ad2c986f07 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:44.125323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5e68ab44-56a8-4345-b0ec-93ad2c986f07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:44.207626 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1319/2640] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:44.229199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e12800-e7a8-4ddd-9e87-73dedae42395 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:44.233682 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-15e12800-e7a8-4ddd-9e87-73dedae42395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:44.358324 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e12800-e7a8-4ddd-9e87-73dedae42395 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:44.363633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6d95c-bb89-483a-a76b-9f29fc167a0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:44.364381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6d95c-bb89-483a-a76b-9f29fc167a0b service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:44.364670 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6d95c-bb89-483a-a76b-9f29fc167a0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:44.364990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6d95c-bb89-483a-a76b-9f29fc167a0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:44.366979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68ab44-56a8-4345-b0ec-93ad2c986f07 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:44.376528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add6d95c-bb89-483a-a76b-9f29fc167a0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:44.378104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add6d95c-bb89-483a-a76b-9f29fc167a0b service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:44.389389 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1320/2641] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:44.390411 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6d95c-bb89-483a-a76b-9f29fc167a0b service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:44.392999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-add6d95c-bb89-483a-a76b-9f29fc167a0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:44.410002 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1b999-37f4-4f5b-b127-41074e89abfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:44.410911 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1b999-37f4-4f5b-b127-41074e89abfa service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:44.412256 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1b999-37f4-4f5b-b127-41074e89abfa service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:44.412743 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1b999-37f4-4f5b-b127-41074e89abfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:44.422246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-b7c1b999-37f4-4f5b-b127-41074e89abfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:44.464707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c1b999-37f4-4f5b-b127-41074e89abfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:44.467075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c1b999-37f4-4f5b-b127-41074e89abfa service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:44.499721 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1322/2642] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:30:44.502442 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6d95c-bb89-483a-a76b-9f29fc167a0b service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:44.518826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934fe99-9b2d-4d15-abe3-9827b9b21c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:44.519726 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934fe99-9b2d-4d15-abe3-9827b9b21c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:44.520008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934fe99-9b2d-4d15-abe3-9827b9b21c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:44.520619 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934fe99-9b2d-4d15-abe3-9827b9b21c41 admin admin] PATH_INFO: `/v3/projects/c03b717751ed4cd9ba8189526da46a70` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:44.526748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934fe99-9b2d-4d15-abe3-9827b9b21c41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c03b717751ed4cd9ba8189526da46a70)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:44.530282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934fe99-9b2d-4d15-abe3-9827b9b21c41 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:44.619630 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:44.625563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9934fe99-9b2d-4d15-abe3-9827b9b21c41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c03b717751ed4cd9ba8189526da46a70', 'request_id': 'req-9934fe99-9b2d-4d15-abe3-9827b9b21c41'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:44.644058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9934fe99-9b2d-4d15-abe3-9827b9b21c41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c03b717751ed4cd9ba8189526da46a70', 'request_id': 'req-9934fe99-9b2d-4d15-abe3-9827b9b21c41'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:44.655628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9934fe99-9b2d-4d15-abe3-9827b9b21c41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c03b717751ed4cd9ba8189526da46a70', 'request_id': 'req-9934fe99-9b2d-4d15-abe3-9827b9b21c41'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:44.671064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9934fe99-9b2d-4d15-abe3-9827b9b21c41 admin admin] The token cache is being invalidate because project c03b717751ed4cd9ba8189526da46a70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:44.671443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9934fe99-9b2d-4d15-abe3-9827b9b21c41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:44.672571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9934fe99-9b2d-4d15-abe3-9827b9b21c41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:44.674737 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1321/2643] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:44 2022] DELETE /identity/v3/projects/c03b717751ed4cd9ba8189526da46a70 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:44.681218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993a809-4cce-4993-8b6b-d00bbd86435f None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:44.682016 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993a809-4cce-4993-8b6b-d00bbd86435f None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:44.682145 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993a809-4cce-4993-8b6b-d00bbd86435f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:44.707725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-4993a809-4cce-4993-8b6b-d00bbd86435f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:44.709412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4993a809-4cce-4993-8b6b-d00bbd86435f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:44.832335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1323/2644] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:44.839821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2a19f-0c26-48e9-a1a9-05f915f90c2e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:44.842132 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2a19f-0c26-48e9-a1a9-05f915f90c2e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:44.842833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2a19f-0c26-48e9-a1a9-05f915f90c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:44.877819 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a6f2a19f-0c26-48e9-a1a9-05f915f90c2e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:44.879285 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a6f2a19f-0c26-48e9-a1a9-05f915f90c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:44.897563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1322/2645] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:44.902311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993a809-4cce-4993-8b6b-d00bbd86435f None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:44.916686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993a809-4cce-4993-8b6b-d00bbd86435f None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:44.923843 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e6be8-35e7-4ad5-a9f2-5d1f3018c707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:44.924547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e6be8-35e7-4ad5-a9f2-5d1f3018c707 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:44.924836 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e6be8-35e7-4ad5-a9f2-5d1f3018c707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:44.925118 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e6be8-35e7-4ad5-a9f2-5d1f3018c707 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:44.925890 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e6be8-35e7-4ad5-a9f2-5d1f3018c707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:30:44.926435 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e6be8-35e7-4ad5-a9f2-5d1f3018c707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:44.927976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e6be8-35e7-4ad5-a9f2-5d1f3018c707 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:44.945282 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1324/2646] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:30:44.946617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2a19f-0c26-48e9-a1a9-05f915f90c2e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:44.962704 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2a19f-0c26-48e9-a1a9-05f915f90c2e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:44.968955 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee414ea-3040-41bd-b9c6-819f71c4748c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:44.969643 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee414ea-3040-41bd-b9c6-819f71c4748c admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:44.969929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee414ea-3040-41bd-b9c6-819f71c4748c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:44.970197 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee414ea-3040-41bd-b9c6-819f71c4748c admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:44.970900 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee414ea-3040-41bd-b9c6-819f71c4748c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:44.973874 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee414ea-3040-41bd-b9c6-819f71c4748c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:45.007772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1323/2647] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:44 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:45.011945 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e6be8-35e7-4ad5-a9f2-5d1f3018c707 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:45.022775 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e6be8-35e7-4ad5-a9f2-5d1f3018c707 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:45.057882 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:45.085599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a35cd-355d-4ec5-a61c-3fd8690d68df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:45.086286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a35cd-355d-4ec5-a61c-3fd8690d68df admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:45.086573 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a35cd-355d-4ec5-a61c-3fd8690d68df admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:45.086850 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a35cd-355d-4ec5-a61c-3fd8690d68df admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:45.087557 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a35cd-355d-4ec5-a61c-3fd8690d68df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:45.089749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a35cd-355d-4ec5-a61c-3fd8690d68df admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:45.108778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1325/2648] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:45 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:45.113455 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee414ea-3040-41bd-b9c6-819f71c4748c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:45.128454 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee414ea-3040-41bd-b9c6-819f71c4748c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:45.133318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538f345-4697-4f0a-ba5b-faedd0479893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:45.134029 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538f345-4697-4f0a-ba5b-faedd0479893 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:45.134319 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538f345-4697-4f0a-ba5b-faedd0479893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:45.134605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538f345-4697-4f0a-ba5b-faedd0479893 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:45.135361 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538f345-4697-4f0a-ba5b-faedd0479893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:45.137022 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538f345-4697-4f0a-ba5b-faedd0479893 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:45.152425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1324/2649] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:30:45.157082 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a35cd-355d-4ec5-a61c-3fd8690d68df admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:45.167359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a35cd-355d-4ec5-a61c-3fd8690d68df admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:45.171989 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3d2fb-900e-46f0-9c4d-f8a048088e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:45.172870 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3d2fb-900e-46f0-9c4d-f8a048088e77 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:45.173156 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3d2fb-900e-46f0-9c4d-f8a048088e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:45.173430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3d2fb-900e-46f0-9c4d-f8a048088e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:45.174159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3d2fb-900e-46f0-9c4d-f8a048088e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:45.175735 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3d2fb-900e-46f0-9c4d-f8a048088e77 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:45.187466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1326/2650] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:45.192449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538f345-4697-4f0a-ba5b-faedd0479893 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:45.205308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538f345-4697-4f0a-ba5b-faedd0479893 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:45.210182 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d5d0b-fa2f-449b-904c-8e64d60b593d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:45.211062 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d5d0b-fa2f-449b-904c-8e64d60b593d admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:45.211339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d5d0b-fa2f-449b-904c-8e64d60b593d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:45.211612 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d5d0b-fa2f-449b-904c-8e64d60b593d admin admin] PATH_INFO: `/v3/projects/307774676e5646698b3e92a87ee01e0b/users/d7ac94c392fa4bd3b9676ee490d06d38/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:45.227497 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d5d0b-fa2f-449b-904c-8e64d60b593d admin admin] RBAC: Authorizing `identity:create_grant(project_id=307774676e5646698b3e92a87ee01e0b, user_id=d7ac94c392fa4bd3b9676ee490d06d38, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:45.230160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d5d0b-fa2f-449b-904c-8e64d60b593d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:45.243316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1325/2651] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:45 2022] PUT /identity/v3/projects/307774676e5646698b3e92a87ee01e0b/users/d7ac94c392fa4bd3b9676ee490d06d38/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:45.255354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168866c-013b-42df-8dfd-c18503abd7e6 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:45.255638 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168866c-013b-42df-8dfd-c18503abd7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:45.255921 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168866c-013b-42df-8dfd-c18503abd7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:45.283362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-9168866c-013b-42df-8dfd-c18503abd7e6 None None] MFA Rules not processed for user `d7ac94c392fa4bd3b9676ee490d06d38`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:45.284607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-9168866c-013b-42df-8dfd-c18503abd7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:45.357254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1327/2652] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:45.361720 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d5d0b-fa2f-449b-904c-8e64d60b593d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:45.435755 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d5d0b-fa2f-449b-904c-8e64d60b593d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:45.440866 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30a29f-ef6c-4227-ba35-06d57c4d9635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:45.441536 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30a29f-ef6c-4227-ba35-06d57c4d9635 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:45.441827 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30a29f-ef6c-4227-ba35-06d57c4d9635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:45.442092 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30a29f-ef6c-4227-ba35-06d57c4d9635 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:45.442793 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30a29f-ef6c-4227-ba35-06d57c4d9635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:45.445033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30a29f-ef6c-4227-ba35-06d57c4d9635 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:45.473004 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1326/2653] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:45 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:30:45.477089 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168866c-013b-42df-8dfd-c18503abd7e6 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:45.489323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168866c-013b-42df-8dfd-c18503abd7e6 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:45.548956 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2f654-4750-4299-93ae-87f24516831b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:45.549645 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2f654-4750-4299-93ae-87f24516831b admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:45.549931 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2f654-4750-4299-93ae-87f24516831b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:45.550210 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2f654-4750-4299-93ae-87f24516831b admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:45.550924 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2f654-4750-4299-93ae-87f24516831b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:45.552867 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2f654-4750-4299-93ae-87f24516831b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:45.579736 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1328/2654] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:45 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:45.580990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30a29f-ef6c-4227-ba35-06d57c4d9635 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:45.591716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30a29f-ef6c-4227-ba35-06d57c4d9635 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:45.599049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624c63a-22ad-490e-a3a0-fc37982239c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:45.599049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624c63a-22ad-490e-a3a0-fc37982239c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:45.599049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624c63a-22ad-490e-a3a0-fc37982239c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:45.599049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624c63a-22ad-490e-a3a0-fc37982239c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:45.609247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624c63a-22ad-490e-a3a0-fc37982239c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:45.609247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624c63a-22ad-490e-a3a0-fc37982239c8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:45.619352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1327/2655] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:45.624358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2f654-4750-4299-93ae-87f24516831b admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:45.642115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2f654-4750-4299-93ae-87f24516831b admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:45.651021 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd4360-b552-4c54-8e30-c4d801c269b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:45.652546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd4360-b552-4c54-8e30-c4d801c269b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:45.653038 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd4360-b552-4c54-8e30-c4d801c269b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:45.653522 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd4360-b552-4c54-8e30-c4d801c269b0 admin admin] PATH_INFO: `/v3/projects/fdbe506b02dc4adfba07da02079c519a/users/ed47648483de4d10b4a65882c32f2e51/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:45.681659 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd4360-b552-4c54-8e30-c4d801c269b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdbe506b02dc4adfba07da02079c519a, user_id=ed47648483de4d10b4a65882c32f2e51, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:45.685771 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd4360-b552-4c54-8e30-c4d801c269b0 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:45.706788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1329/2656] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:45 2022] PUT /identity/v3/projects/fdbe506b02dc4adfba07da02079c519a/users/ed47648483de4d10b4a65882c32f2e51/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:45.711178 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624c63a-22ad-490e-a3a0-fc37982239c8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:45.728591 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624c63a-22ad-490e-a3a0-fc37982239c8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:45.786481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:45.805736 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09fec47-2571-4447-9d24-5d571779f607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:45.806425 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fec47-2571-4447-9d24-5d571779f607 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:45.806714 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fec47-2571-4447-9d24-5d571779f607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:45.806994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fec47-2571-4447-9d24-5d571779f607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:45.807813 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09fec47-2571-4447-9d24-5d571779f607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:45.809454 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09fec47-2571-4447-9d24-5d571779f607 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:45.819893 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1328/2657] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:45.824013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd4360-b552-4c54-8e30-c4d801c269b0 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:45.834300 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd4360-b552-4c54-8e30-c4d801c269b0 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:45.838961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db386b8-ea7d-4f5a-9219-9603cba6e744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:45.839824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db386b8-ea7d-4f5a-9219-9603cba6e744 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:45.840100 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db386b8-ea7d-4f5a-9219-9603cba6e744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:45.840912 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db386b8-ea7d-4f5a-9219-9603cba6e744 admin admin] PATH_INFO: `/v3/projects/fdbe506b02dc4adfba07da02079c519a/users/ed47648483de4d10b4a65882c32f2e51/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:45.842859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db386b8-ea7d-4f5a-9219-9603cba6e744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdbe506b02dc4adfba07da02079c519a, user_id=ed47648483de4d10b4a65882c32f2e51, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:45.845535 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db386b8-ea7d-4f5a-9219-9603cba6e744 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:45.851012 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:45.857172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1330/2658] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:45 2022] PUT /identity/v3/projects/fdbe506b02dc4adfba07da02079c519a/users/ed47648483de4d10b4a65882c32f2e51/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:45.867734 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3be62b-9243-445c-a387-e55bb2bdbbae None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:45.868028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3be62b-9243-445c-a387-e55bb2bdbbae None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:45.868335 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3be62b-9243-445c-a387-e55bb2bdbbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:45.895534 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8a3be62b-9243-445c-a387-e55bb2bdbbae None None] MFA Rules not processed for user `ed47648483de4d10b4a65882c32f2e51`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:45.896829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8a3be62b-9243-445c-a387-e55bb2bdbbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:45.952764 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb8f48-29cf-4b78-9a41-b6b77926d122 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:45.953189 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb8f48-29cf-4b78-9a41-b6b77926d122 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:45.953581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb8f48-29cf-4b78-9a41-b6b77926d122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:45.972795 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-83cb8f48-29cf-4b78-9a41-b6b77926d122 None None] MFA Rules not processed for user `44d26a97291e44c78d3a6438d7ab76c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:45.974018 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-83cb8f48-29cf-4b78-9a41-b6b77926d122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:45.975997 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1329/2659] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:45.989142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3be62b-9243-445c-a387-e55bb2bdbbae None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:45.990670 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8a3be62b-9243-445c-a387-e55bb2bdbbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:46.059182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1331/2660] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:46.063324 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3be62b-9243-445c-a387-e55bb2bdbbae None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:46.067275 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a074eb-e88c-4db3-91aa-b8d47cf836f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:46.067924 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a074eb-e88c-4db3-91aa-b8d47cf836f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:46.068213 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a074eb-e88c-4db3-91aa-b8d47cf836f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:46.068536 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a074eb-e88c-4db3-91aa-b8d47cf836f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:46.078603 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a074eb-e88c-4db3-91aa-b8d47cf836f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:46.080053 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a074eb-e88c-4db3-91aa-b8d47cf836f3 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:46.093728 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1330/2661] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:30:46.184777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb8f48-29cf-4b78-9a41-b6b77926d122 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:46.200586 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb8f48-29cf-4b78-9a41-b6b77926d122 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:46.204532 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45371f-0bbd-4e41-b724-9397c7bd61b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:46.205341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45371f-0bbd-4e41-b724-9397c7bd61b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:46.205824 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45371f-0bbd-4e41-b724-9397c7bd61b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:46.206105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45371f-0bbd-4e41-b724-9397c7bd61b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:46.270347 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45371f-0bbd-4e41-b724-9397c7bd61b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:46.271838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45371f-0bbd-4e41-b724-9397c7bd61b8 service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:46.281799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1332/2662] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:46.584646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a074eb-e88c-4db3-91aa-b8d47cf836f3 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:46.678211 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a074eb-e88c-4db3-91aa-b8d47cf836f3 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:46.688655 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1fed07-23be-4cf6-9d56-3790670c8199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:46.689582 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fed07-23be-4cf6-9d56-3790670c8199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:46.689981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fed07-23be-4cf6-9d56-3790670c8199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:46.690514 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fed07-23be-4cf6-9d56-3790670c8199 admin admin] PATH_INFO: `/v3/users/d7ac94c392fa4bd3b9676ee490d06d38` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:46.691739 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1fed07-23be-4cf6-9d56-3790670c8199 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7ac94c392fa4bd3b9676ee490d06d38)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:46.693814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1fed07-23be-4cf6-9d56-3790670c8199 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:46.705053 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:46.756079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ca1fed07-23be-4cf6-9d56-3790670c8199 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7ac94c392fa4bd3b9676ee490d06d38'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:46.764762 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ca1fed07-23be-4cf6-9d56-3790670c8199 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7ac94c392fa4bd3b9676ee490d06d38'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:46.765418 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ca1fed07-23be-4cf6-9d56-3790670c8199 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7ac94c392fa4bd3b9676ee490d06d38'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:46.773232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ca1fed07-23be-4cf6-9d56-3790670c8199 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7ac94c392fa4bd3b9676ee490d06d38'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:46.787230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ca1fed07-23be-4cf6-9d56-3790670c8199 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7ac94c392fa4bd3b9676ee490d06d38'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:46.796151 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1331/2663] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:46 2022] DELETE /identity/v3/users/d7ac94c392fa4bd3b9676ee490d06d38 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:46.804634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf39de-6a04-4d8d-90ac-26859a774247 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:46.805382 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf39de-6a04-4d8d-90ac-26859a774247 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:46.808722 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf39de-6a04-4d8d-90ac-26859a774247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:46.839065 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-00bf39de-6a04-4d8d-90ac-26859a774247 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:46.841533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-00bf39de-6a04-4d8d-90ac-26859a774247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:46.947112 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1333/2664] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:46.947998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1fed07-23be-4cf6-9d56-3790670c8199 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:46.949344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ca1fed07-23be-4cf6-9d56-3790670c8199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:46.953168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf39de-6a04-4d8d-90ac-26859a774247 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:46.954432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-00bf39de-6a04-4d8d-90ac-26859a774247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:47.036207 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1fed07-23be-4cf6-9d56-3790670c8199 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:47.040253 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47ebd9-35f4-4546-a254-2ddf894d29d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:47.040840 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47ebd9-35f4-4546-a254-2ddf894d29d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:47.041122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47ebd9-35f4-4546-a254-2ddf894d29d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:47.041403 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47ebd9-35f4-4546-a254-2ddf894d29d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:47.043474 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bd47ebd9-35f4-4546-a254-2ddf894d29d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:47.044762 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=100 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:47.061071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47ebd9-35f4-4546-a254-2ddf894d29d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:47.063059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47ebd9-35f4-4546-a254-2ddf894d29d8 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:47.063286 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf39de-6a04-4d8d-90ac-26859a774247 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:47.067504 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e7527-9058-4852-a92a-d0a8c1034627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:47.068175 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e7527-9058-4852-a92a-d0a8c1034627 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:47.068492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e7527-9058-4852-a92a-d0a8c1034627 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:47.068766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e7527-9058-4852-a92a-d0a8c1034627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:47.081127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e7527-9058-4852-a92a-d0a8c1034627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:47.082736 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e7527-9058-4852-a92a-d0a8c1034627 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:47.087284 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1332/2665] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:30:47.099264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1334/2666] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:30:47.520539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47ebd9-35f4-4546-a254-2ddf894d29d8 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:47.522017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bd47ebd9-35f4-4546-a254-2ddf894d29d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:47.538209 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47ebd9-35f4-4546-a254-2ddf894d29d8 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:47.542487 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67cad9-06d3-4c83-8e22-e7fac73b2365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:47.543437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67cad9-06d3-4c83-8e22-e7fac73b2365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:47.543933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67cad9-06d3-4c83-8e22-e7fac73b2365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:47.544443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67cad9-06d3-4c83-8e22-e7fac73b2365 admin admin] PATH_INFO: `/v3/projects/307774676e5646698b3e92a87ee01e0b` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:47.545727 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67cad9-06d3-4c83-8e22-e7fac73b2365 admin admin] RBAC: Authorizing `identity:delete_project(project_id=307774676e5646698b3e92a87ee01e0b)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:47.547766 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67cad9-06d3-4c83-8e22-e7fac73b2365 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:47.600855 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-bb67cad9-06d3-4c83-8e22-e7fac73b2365 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '307774676e5646698b3e92a87ee01e0b', 'request_id': 'req-bb67cad9-06d3-4c83-8e22-e7fac73b2365'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:47.607230 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-bb67cad9-06d3-4c83-8e22-e7fac73b2365 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '307774676e5646698b3e92a87ee01e0b', 'request_id': 'req-bb67cad9-06d3-4c83-8e22-e7fac73b2365'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:47.620830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-bb67cad9-06d3-4c83-8e22-e7fac73b2365 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '307774676e5646698b3e92a87ee01e0b', 'request_id': 'req-bb67cad9-06d3-4c83-8e22-e7fac73b2365'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:47.638323 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-bb67cad9-06d3-4c83-8e22-e7fac73b2365 admin admin] The token cache is being invalidate because project 307774676e5646698b3e92a87ee01e0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:47.638651 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-bb67cad9-06d3-4c83-8e22-e7fac73b2365 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:47.639314 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-bb67cad9-06d3-4c83-8e22-e7fac73b2365 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:47.640908 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1333/2667] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:47 2022] DELETE /identity/v3/projects/307774676e5646698b3e92a87ee01e0b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:30:47.650466 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e7527-9058-4852-a92a-d0a8c1034627 service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:47.663001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e7527-9058-4852-a92a-d0a8c1034627 service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:47.775401 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27722d1-80bd-42ae-b62f-2f35e53503be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:47.776402 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27722d1-80bd-42ae-b62f-2f35e53503be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:47.776689 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27722d1-80bd-42ae-b62f-2f35e53503be admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:47.777089 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27722d1-80bd-42ae-b62f-2f35e53503be admin admin] PATH_INFO: `/v3/users/ed47648483de4d10b4a65882c32f2e51` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:47.779839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27722d1-80bd-42ae-b62f-2f35e53503be admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed47648483de4d10b4a65882c32f2e51)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:47.781897 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27722d1-80bd-42ae-b62f-2f35e53503be admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:47.919159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b27722d1-80bd-42ae-b62f-2f35e53503be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed47648483de4d10b4a65882c32f2e51'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:47.930819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b27722d1-80bd-42ae-b62f-2f35e53503be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed47648483de4d10b4a65882c32f2e51'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:47.950527 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b27722d1-80bd-42ae-b62f-2f35e53503be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed47648483de4d10b4a65882c32f2e51'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:47.968741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b27722d1-80bd-42ae-b62f-2f35e53503be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed47648483de4d10b4a65882c32f2e51'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:47.969633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-b27722d1-80bd-42ae-b62f-2f35e53503be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed47648483de4d10b4a65882c32f2e51'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:47.976647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1335/2668] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:47 2022] DELETE /identity/v3/users/ed47648483de4d10b4a65882c32f2e51 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:48.514252 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67cad9-06d3-4c83-8e22-e7fac73b2365 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:48.518005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bb67cad9-06d3-4c83-8e22-e7fac73b2365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:48.541557 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67cad9-06d3-4c83-8e22-e7fac73b2365 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:48.573945 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:48.682287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c03b2-74f1-4029-b70a-84717d12ba68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:48.692879 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c03b2-74f1-4029-b70a-84717d12ba68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:48.693510 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c03b2-74f1-4029-b70a-84717d12ba68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:48.693951 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c03b2-74f1-4029-b70a-84717d12ba68 admin admin] PATH_INFO: `/v3/projects/fdbe506b02dc4adfba07da02079c519a` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:48.696133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c03b2-74f1-4029-b70a-84717d12ba68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdbe506b02dc4adfba07da02079c519a)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:48.698830 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c03b2-74f1-4029-b70a-84717d12ba68 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:48.781517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f73c03b2-74f1-4029-b70a-84717d12ba68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdbe506b02dc4adfba07da02079c519a', 'request_id': 'req-f73c03b2-74f1-4029-b70a-84717d12ba68'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:48.791820 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f73c03b2-74f1-4029-b70a-84717d12ba68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdbe506b02dc4adfba07da02079c519a', 'request_id': 'req-f73c03b2-74f1-4029-b70a-84717d12ba68'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:48.808027 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f73c03b2-74f1-4029-b70a-84717d12ba68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdbe506b02dc4adfba07da02079c519a', 'request_id': 'req-f73c03b2-74f1-4029-b70a-84717d12ba68'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:48.823826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f73c03b2-74f1-4029-b70a-84717d12ba68 admin admin] The token cache is being invalidate because project fdbe506b02dc4adfba07da02079c519a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:48.824521 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f73c03b2-74f1-4029-b70a-84717d12ba68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:48.827176 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f73c03b2-74f1-4029-b70a-84717d12ba68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:48.832193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1334/2669] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:48 2022] DELETE /identity/v3/projects/fdbe506b02dc4adfba07da02079c519a => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:48.834947 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71f34e-70fc-4f1d-ad5f-491866497c0e None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:48.835247 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71f34e-70fc-4f1d-ad5f-491866497c0e None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:48.835536 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71f34e-70fc-4f1d-ad5f-491866497c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:48.872154 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-1d71f34e-70fc-4f1d-ad5f-491866497c0e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:48.873460 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-1d71f34e-70fc-4f1d-ad5f-491866497c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:48.993492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1336/2670] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:49.002034 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6618a-71bb-4842-bbdb-c5498e6d2950 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:49.002434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6618a-71bb-4842-bbdb-c5498e6d2950 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:49.002700 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6618a-71bb-4842-bbdb-c5498e6d2950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:49.031386 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4bb6618a-71bb-4842-bbdb-c5498e6d2950 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:49.032679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4bb6618a-71bb-4842-bbdb-c5498e6d2950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:49.051751 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1335/2671] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:49.055625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71f34e-70fc-4f1d-ad5f-491866497c0e None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:49.062540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:49.068092 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71f34e-70fc-4f1d-ad5f-491866497c0e None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:49.075223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07deae0a-347b-4ce6-8c05-ff9cd4a0ad5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:49.076054 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07deae0a-347b-4ce6-8c05-ff9cd4a0ad5b admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:49.076753 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07deae0a-347b-4ce6-8c05-ff9cd4a0ad5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:49.077758 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07deae0a-347b-4ce6-8c05-ff9cd4a0ad5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:49.079538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07deae0a-347b-4ce6-8c05-ff9cd4a0ad5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:30:49.079538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07deae0a-347b-4ce6-8c05-ff9cd4a0ad5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:49.081251 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07deae0a-347b-4ce6-8c05-ff9cd4a0ad5b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:49.095849 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1337/2672] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:30:49.100046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6618a-71bb-4842-bbdb-c5498e6d2950 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:49.116674 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6618a-71bb-4842-bbdb-c5498e6d2950 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:49.121363 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fe417-23c0-44b8-815d-832873e1cc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:49.122668 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fe417-23c0-44b8-815d-832873e1cc58 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:49.122961 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fe417-23c0-44b8-815d-832873e1cc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:49.123239 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fe417-23c0-44b8-815d-832873e1cc58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:49.123983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fe417-23c0-44b8-815d-832873e1cc58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:49.125977 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fe417-23c0-44b8-815d-832873e1cc58 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:49.160764 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07deae0a-347b-4ce6-8c05-ff9cd4a0ad5b admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:49.168809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1336/2673] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:49 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:30:49.173893 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07deae0a-347b-4ce6-8c05-ff9cd4a0ad5b admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:49.245218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b1e7d-6fcb-42aa-93a4-57cfb9cbbb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:49.246507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b1e7d-6fcb-42aa-93a4-57cfb9cbbb7d admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:49.246507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b1e7d-6fcb-42aa-93a4-57cfb9cbbb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:49.246507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b1e7d-6fcb-42aa-93a4-57cfb9cbbb7d admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:49.247854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b1e7d-6fcb-42aa-93a4-57cfb9cbbb7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:49.249818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b1e7d-6fcb-42aa-93a4-57cfb9cbbb7d admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:49.277687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1338/2674] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:49 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:30:49.282091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fe417-23c0-44b8-815d-832873e1cc58 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:49.297839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fe417-23c0-44b8-815d-832873e1cc58 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:49.300382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255b6e8-a7e5-4742-a640-b3a2db7056c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:49.300831 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255b6e8-a7e5-4742-a640-b3a2db7056c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:49.301115 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255b6e8-a7e5-4742-a640-b3a2db7056c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:49.301392 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255b6e8-a7e5-4742-a640-b3a2db7056c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:49.302136 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255b6e8-a7e5-4742-a640-b3a2db7056c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:49.303796 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255b6e8-a7e5-4742-a640-b3a2db7056c7 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:49.318261 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1337/2675] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:49.319746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b1e7d-6fcb-42aa-93a4-57cfb9cbbb7d admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:49.331299 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b1e7d-6fcb-42aa-93a4-57cfb9cbbb7d admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:49.335871 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356de73-0e63-4373-a1ff-a5186f16c69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:49.337075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356de73-0e63-4373-a1ff-a5186f16c69e admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:49.337075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356de73-0e63-4373-a1ff-a5186f16c69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:49.337263 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356de73-0e63-4373-a1ff-a5186f16c69e admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:49.337990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8356de73-0e63-4373-a1ff-a5186f16c69e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:49.339612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8356de73-0e63-4373-a1ff-a5186f16c69e admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:49.357028 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1339/2676] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:49.361033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255b6e8-a7e5-4742-a640-b3a2db7056c7 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:49.371624 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255b6e8-a7e5-4742-a640-b3a2db7056c7 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:49.380926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16638e9-7177-46b4-a2d6-0a3343fb7c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:49.381839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16638e9-7177-46b4-a2d6-0a3343fb7c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:49.382123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16638e9-7177-46b4-a2d6-0a3343fb7c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:49.382451 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16638e9-7177-46b4-a2d6-0a3343fb7c70 admin admin] PATH_INFO: `/v3/projects/a96e15b06ef34521834b1a93afdfd434/users/42be4d3c31834a81b2417e7348c3bab1/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:49.405947 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16638e9-7177-46b4-a2d6-0a3343fb7c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a96e15b06ef34521834b1a93afdfd434, user_id=42be4d3c31834a81b2417e7348c3bab1, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:49.408802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16638e9-7177-46b4-a2d6-0a3343fb7c70 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:49.451388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1338/2677] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:49 2022] PUT /identity/v3/projects/a96e15b06ef34521834b1a93afdfd434/users/42be4d3c31834a81b2417e7348c3bab1/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:49.454742 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3f440-8a27-47fc-bf2e-16f419e6a09b None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:49.455085 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3f440-8a27-47fc-bf2e-16f419e6a09b None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:49.455323 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3f440-8a27-47fc-bf2e-16f419e6a09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:49.487082 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-a3b3f440-8a27-47fc-bf2e-16f419e6a09b None None] MFA Rules not processed for user `42be4d3c31834a81b2417e7348c3bab1`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:49.488354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-a3b3f440-8a27-47fc-bf2e-16f419e6a09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:49.580519 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1340/2678] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:49.596226 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16638e9-7177-46b4-a2d6-0a3343fb7c70 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:49.600959 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b16638e9-7177-46b4-a2d6-0a3343fb7c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:49.651291 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:49.676421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16638e9-7177-46b4-a2d6-0a3343fb7c70 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:49.680724 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3df40b-7b83-4d11-b93c-e95a58fb3eb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:49.681386 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3df40b-7b83-4d11-b93c-e95a58fb3eb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:49.681670 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3df40b-7b83-4d11-b93c-e95a58fb3eb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:49.681944 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3df40b-7b83-4d11-b93c-e95a58fb3eb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:49.692546 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3df40b-7b83-4d11-b93c-e95a58fb3eb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:49.694308 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3df40b-7b83-4d11-b93c-e95a58fb3eb5 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:49.704933 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1339/2679] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:49.730052 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3f440-8a27-47fc-bf2e-16f419e6a09b None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:49.823565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3f440-8a27-47fc-bf2e-16f419e6a09b None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:49.829846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085149ea-f68f-402d-8758-5d7273e11c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:49.830825 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085149ea-f68f-402d-8758-5d7273e11c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:49.831046 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085149ea-f68f-402d-8758-5d7273e11c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:49.831329 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085149ea-f68f-402d-8758-5d7273e11c7c admin admin] PATH_INFO: `/v3/users/42be4d3c31834a81b2417e7348c3bab1` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:49.833078 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085149ea-f68f-402d-8758-5d7273e11c7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=42be4d3c31834a81b2417e7348c3bab1)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:49.835036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085149ea-f68f-402d-8758-5d7273e11c7c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:49.836876 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:49.895407 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-085149ea-f68f-402d-8758-5d7273e11c7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42be4d3c31834a81b2417e7348c3bab1'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:49.905040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-085149ea-f68f-402d-8758-5d7273e11c7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42be4d3c31834a81b2417e7348c3bab1'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:49.920311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-085149ea-f68f-402d-8758-5d7273e11c7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42be4d3c31834a81b2417e7348c3bab1'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:49.927574 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-085149ea-f68f-402d-8758-5d7273e11c7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42be4d3c31834a81b2417e7348c3bab1'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:49.928540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-085149ea-f68f-402d-8758-5d7273e11c7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42be4d3c31834a81b2417e7348c3bab1'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:49.936659 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1341/2680] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:49 2022] DELETE /identity/v3/users/42be4d3c31834a81b2417e7348c3bab1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:49.942486 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4749773-ec00-4b7b-a5a3-6ed5fda1ebcf None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:49.943006 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4749773-ec00-4b7b-a5a3-6ed5fda1ebcf None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:49.943475 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4749773-ec00-4b7b-a5a3-6ed5fda1ebcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:49.966405 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a4749773-ec00-4b7b-a5a3-6ed5fda1ebcf None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:49.968039 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a4749773-ec00-4b7b-a5a3-6ed5fda1ebcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:50.040487 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1340/2681] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:50.050885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085149ea-f68f-402d-8758-5d7273e11c7c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:50.052764 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-085149ea-f68f-402d-8758-5d7273e11c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:50.138843 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085149ea-f68f-402d-8758-5d7273e11c7c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:50.145617 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c407697-5b89-416c-b0ca-b4cc49bfde56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:50.147182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c407697-5b89-416c-b0ca-b4cc49bfde56 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:50.147658 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c407697-5b89-416c-b0ca-b4cc49bfde56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:50.148124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c407697-5b89-416c-b0ca-b4cc49bfde56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:50.161341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c407697-5b89-416c-b0ca-b4cc49bfde56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:50.163443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c407697-5b89-416c-b0ca-b4cc49bfde56 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:50.191087 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1342/2682] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:50.497420 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4749773-ec00-4b7b-a5a3-6ed5fda1ebcf None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:50.498891 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a4749773-ec00-4b7b-a5a3-6ed5fda1ebcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:50.512772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4749773-ec00-4b7b-a5a3-6ed5fda1ebcf None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:50.529416 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929072f-06f7-4994-adc2-5608ad63fa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:50.530824 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929072f-06f7-4994-adc2-5608ad63fa8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:50.531112 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929072f-06f7-4994-adc2-5608ad63fa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:50.531395 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929072f-06f7-4994-adc2-5608ad63fa8f admin admin] PATH_INFO: `/v3/projects/a96e15b06ef34521834b1a93afdfd434` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:50.532678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929072f-06f7-4994-adc2-5608ad63fa8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a96e15b06ef34521834b1a93afdfd434)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:50.534599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929072f-06f7-4994-adc2-5608ad63fa8f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:50.591690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a929072f-06f7-4994-adc2-5608ad63fa8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a96e15b06ef34521834b1a93afdfd434', 'request_id': 'req-a929072f-06f7-4994-adc2-5608ad63fa8f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:50.601592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a929072f-06f7-4994-adc2-5608ad63fa8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a96e15b06ef34521834b1a93afdfd434', 'request_id': 'req-a929072f-06f7-4994-adc2-5608ad63fa8f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:50.613278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a929072f-06f7-4994-adc2-5608ad63fa8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a96e15b06ef34521834b1a93afdfd434', 'request_id': 'req-a929072f-06f7-4994-adc2-5608ad63fa8f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:50.626434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a929072f-06f7-4994-adc2-5608ad63fa8f admin admin] The token cache is being invalidate because project a96e15b06ef34521834b1a93afdfd434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:50.626434 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a929072f-06f7-4994-adc2-5608ad63fa8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:50.627166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a929072f-06f7-4994-adc2-5608ad63fa8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:50.629060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1341/2683] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:50 2022] DELETE /identity/v3/projects/a96e15b06ef34521834b1a93afdfd434 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:50.634315 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e45d17-735c-440f-8d0d-90a0dde40dd5 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:50.634663 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e45d17-735c-440f-8d0d-90a0dde40dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:50.635016 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e45d17-735c-440f-8d0d-90a0dde40dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:50.658889 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-13e45d17-735c-440f-8d0d-90a0dde40dd5 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:50.660128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-13e45d17-735c-440f-8d0d-90a0dde40dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:50.740942 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1343/2684] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:50.749309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b74748-4c12-4e3e-81c1-fc5c41c11468 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:50.749765 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b74748-4c12-4e3e-81c1-fc5c41c11468 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:50.749860 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b74748-4c12-4e3e-81c1-fc5c41c11468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:50.778984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-80b74748-4c12-4e3e-81c1-fc5c41c11468 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:50.780404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-80b74748-4c12-4e3e-81c1-fc5c41c11468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:50.790563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1342/2685] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:50.795875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e45d17-735c-440f-8d0d-90a0dde40dd5 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:50.810177 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e45d17-735c-440f-8d0d-90a0dde40dd5 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:50.816195 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859bdcc-8800-49a5-ab7f-11d4bffd2dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:50.816903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859bdcc-8800-49a5-ab7f-11d4bffd2dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:50.817192 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859bdcc-8800-49a5-ab7f-11d4bffd2dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:50.817472 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859bdcc-8800-49a5-ab7f-11d4bffd2dc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:50.818225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859bdcc-8800-49a5-ab7f-11d4bffd2dc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:30:50.818540 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859bdcc-8800-49a5-ab7f-11d4bffd2dc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:50.820193 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859bdcc-8800-49a5-ab7f-11d4bffd2dc1 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:50.832550 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1344/2686] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:30:50.837871 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b74748-4c12-4e3e-81c1-fc5c41c11468 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:50.849479 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b74748-4c12-4e3e-81c1-fc5c41c11468 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:50.854473 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f38382-b476-4527-8187-bd815b311f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:50.855310 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f38382-b476-4527-8187-bd815b311f65 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:50.856417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f38382-b476-4527-8187-bd815b311f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:50.856620 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f38382-b476-4527-8187-bd815b311f65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:50.857344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f38382-b476-4527-8187-bd815b311f65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:50.859480 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f38382-b476-4527-8187-bd815b311f65 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:50.888577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1343/2687] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:50 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:50.893002 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859bdcc-8800-49a5-ab7f-11d4bffd2dc1 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:50.903705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859bdcc-8800-49a5-ab7f-11d4bffd2dc1 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:50.908922 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:50.991747 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fe011-009e-46b5-bf89-58552db91a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:50.992566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fe011-009e-46b5-bf89-58552db91a41 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:50.992917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fe011-009e-46b5-bf89-58552db91a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:50.993194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fe011-009e-46b5-bf89-58552db91a41 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:50.996333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6fe011-009e-46b5-bf89-58552db91a41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:50.999373 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6fe011-009e-46b5-bf89-58552db91a41 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:51.032963 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1345/2688] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:50 2022] POST /identity/v3/users => generated 296 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:30:51.033679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f38382-b476-4527-8187-bd815b311f65 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:51.050825 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f38382-b476-4527-8187-bd815b311f65 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:51.059125 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3f652-041c-444a-bf14-f8333bcb7922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:51.060050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3f652-041c-444a-bf14-f8333bcb7922 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:51.060451 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3f652-041c-444a-bf14-f8333bcb7922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:51.060734 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3f652-041c-444a-bf14-f8333bcb7922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:51.061500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3f652-041c-444a-bf14-f8333bcb7922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:51.063170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3f652-041c-444a-bf14-f8333bcb7922 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:51.081983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1344/2689] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:51.086066 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fe011-009e-46b5-bf89-58552db91a41 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:51.099272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fe011-009e-46b5-bf89-58552db91a41 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:51.104597 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17572114-36d4-4c4e-a4b3-b14ec7c270c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:51.105988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17572114-36d4-4c4e-a4b3-b14ec7c270c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:51.106909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17572114-36d4-4c4e-a4b3-b14ec7c270c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:51.107204 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17572114-36d4-4c4e-a4b3-b14ec7c270c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:51.107965 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17572114-36d4-4c4e-a4b3-b14ec7c270c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:51.110077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17572114-36d4-4c4e-a4b3-b14ec7c270c7 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:51.125891 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1346/2690] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:51.129801 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3f652-041c-444a-bf14-f8333bcb7922 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:51.144511 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3f652-041c-444a-bf14-f8333bcb7922 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:51.150070 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121549c-2522-4396-aec5-87fdc5a701be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:51.150974 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121549c-2522-4396-aec5-87fdc5a701be admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:51.151288 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121549c-2522-4396-aec5-87fdc5a701be admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:51.151567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121549c-2522-4396-aec5-87fdc5a701be admin admin] PATH_INFO: `/v3/projects/f6b182898e7b4661b59e3c5ddaec19d4/users/a39555d792b64d23b425e1ec77ed50a0/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:51.178569 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121549c-2522-4396-aec5-87fdc5a701be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6b182898e7b4661b59e3c5ddaec19d4, user_id=a39555d792b64d23b425e1ec77ed50a0, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:51.182119 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121549c-2522-4396-aec5-87fdc5a701be admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:51.225079 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1345/2691] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:51 2022] PUT /identity/v3/projects/f6b182898e7b4661b59e3c5ddaec19d4/users/a39555d792b64d23b425e1ec77ed50a0/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:51.298529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e5130-8ebc-478d-b642-c02f57941d7b None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:51.298529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e5130-8ebc-478d-b642-c02f57941d7b None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:51.298529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e5130-8ebc-478d-b642-c02f57941d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:51.334601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-543e5130-8ebc-478d-b642-c02f57941d7b None None] MFA Rules not processed for user `a39555d792b64d23b425e1ec77ed50a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:51.336065 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-543e5130-8ebc-478d-b642-c02f57941d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:51.432940 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1347/2692] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:51.443324 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121549c-2522-4396-aec5-87fdc5a701be admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:51.453299 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b121549c-2522-4396-aec5-87fdc5a701be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:51.466904 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:51.563033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121549c-2522-4396-aec5-87fdc5a701be admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:51.564144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1248f-e653-4fab-a81b-925b6e955330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:51.564998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1248f-e653-4fab-a81b-925b6e955330 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:51.566000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1248f-e653-4fab-a81b-925b6e955330 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:51.566586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1248f-e653-4fab-a81b-925b6e955330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:51.579154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1248f-e653-4fab-a81b-925b6e955330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:51.580931 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1248f-e653-4fab-a81b-925b6e955330 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:51.598146 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1346/2693] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:52.567610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e5130-8ebc-478d-b642-c02f57941d7b None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:52.570396 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-543e5130-8ebc-478d-b642-c02f57941d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:52.645586 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e5130-8ebc-478d-b642-c02f57941d7b None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:52.650168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc9d3b-2b1f-4ed7-bb59-e758ab11eb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:52.651406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc9d3b-2b1f-4ed7-bb59-e758ab11eb85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:52.651830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc9d3b-2b1f-4ed7-bb59-e758ab11eb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:52.652119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc9d3b-2b1f-4ed7-bb59-e758ab11eb85 admin admin] PATH_INFO: `/v3/users/9ac790383bdf4eecaef96cc25fe38553` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:52.653421 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc9d3b-2b1f-4ed7-bb59-e758ab11eb85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ac790383bdf4eecaef96cc25fe38553)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:52.655345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc9d3b-2b1f-4ed7-bb59-e758ab11eb85 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:52.678975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:52.706500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-7edc9d3b-2b1f-4ed7-bb59-e758ab11eb85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ac790383bdf4eecaef96cc25fe38553'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:52.714779 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-7edc9d3b-2b1f-4ed7-bb59-e758ab11eb85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ac790383bdf4eecaef96cc25fe38553'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:52.727769 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-7edc9d3b-2b1f-4ed7-bb59-e758ab11eb85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ac790383bdf4eecaef96cc25fe38553'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:52.734461 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-7edc9d3b-2b1f-4ed7-bb59-e758ab11eb85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ac790383bdf4eecaef96cc25fe38553'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:52.735310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-7edc9d3b-2b1f-4ed7-bb59-e758ab11eb85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ac790383bdf4eecaef96cc25fe38553'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:52.741350 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1348/2694] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:52 2022] DELETE /identity/v3/users/9ac790383bdf4eecaef96cc25fe38553 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:52.915261 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1248f-e653-4fab-a81b-925b6e955330 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:52.932009 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1248f-e653-4fab-a81b-925b6e955330 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:53.003553 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7e8ed-3ab8-4dcb-a488-01dc4001dc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:53.004608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7e8ed-3ab8-4dcb-a488-01dc4001dc19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:53.004919 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7e8ed-3ab8-4dcb-a488-01dc4001dc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:53.005202 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7e8ed-3ab8-4dcb-a488-01dc4001dc19 admin admin] PATH_INFO: `/v3/projects/ff9c443652a141d482130869ff76c327` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:53.006616 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7e8ed-3ab8-4dcb-a488-01dc4001dc19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff9c443652a141d482130869ff76c327)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:53.008695 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7e8ed-3ab8-4dcb-a488-01dc4001dc19 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:53.069280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2eb7e8ed-3ab8-4dcb-a488-01dc4001dc19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff9c443652a141d482130869ff76c327', 'request_id': 'req-2eb7e8ed-3ab8-4dcb-a488-01dc4001dc19'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:53.076754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2eb7e8ed-3ab8-4dcb-a488-01dc4001dc19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff9c443652a141d482130869ff76c327', 'request_id': 'req-2eb7e8ed-3ab8-4dcb-a488-01dc4001dc19'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:53.086583 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2eb7e8ed-3ab8-4dcb-a488-01dc4001dc19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff9c443652a141d482130869ff76c327', 'request_id': 'req-2eb7e8ed-3ab8-4dcb-a488-01dc4001dc19'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:53.099804 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2eb7e8ed-3ab8-4dcb-a488-01dc4001dc19 admin admin] The token cache is being invalidate because project ff9c443652a141d482130869ff76c327 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:53.100201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2eb7e8ed-3ab8-4dcb-a488-01dc4001dc19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:53.100979 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2eb7e8ed-3ab8-4dcb-a488-01dc4001dc19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:53.103278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1347/2695] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:52 2022] DELETE /identity/v3/projects/ff9c443652a141d482130869ff76c327 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:53.108715 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07038966-5730-4587-b057-57c9da423215 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:53.109013 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07038966-5730-4587-b057-57c9da423215 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:53.109302 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07038966-5730-4587-b057-57c9da423215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:53.128939 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-07038966-5730-4587-b057-57c9da423215 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:53.130151 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-07038966-5730-4587-b057-57c9da423215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:53.216910 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1349/2696] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:53.219108 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4fd5f-6cea-4368-b09e-d0e7eec5c9cb None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:53.219317 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4fd5f-6cea-4368-b09e-d0e7eec5c9cb None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:53.219604 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4fd5f-6cea-4368-b09e-d0e7eec5c9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:53.240729 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5ce4fd5f-6cea-4368-b09e-d0e7eec5c9cb None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:53.241840 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5ce4fd5f-6cea-4368-b09e-d0e7eec5c9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:53.251379 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1348/2697] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:53.256969 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07038966-5730-4587-b057-57c9da423215 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:53.267699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07038966-5730-4587-b057-57c9da423215 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:53.274267 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c8d55-9e4a-4c61-b8da-fda0816e43ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:53.274553 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c8d55-9e4a-4c61-b8da-fda0816e43ff admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:53.274829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c8d55-9e4a-4c61-b8da-fda0816e43ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:53.275239 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c8d55-9e4a-4c61-b8da-fda0816e43ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:53.275906 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c8d55-9e4a-4c61-b8da-fda0816e43ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:30:53.276257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c8d55-9e4a-4c61-b8da-fda0816e43ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:53.278400 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c8d55-9e4a-4c61-b8da-fda0816e43ff admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:53.293090 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1350/2698] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:30:53.299005 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4fd5f-6cea-4368-b09e-d0e7eec5c9cb None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:53.309680 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4fd5f-6cea-4368-b09e-d0e7eec5c9cb None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:53.314470 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cd53d-87b2-4cbc-a40b-e6e11218f608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:53.315370 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cd53d-87b2-4cbc-a40b-e6e11218f608 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:53.315658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cd53d-87b2-4cbc-a40b-e6e11218f608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:53.315938 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cd53d-87b2-4cbc-a40b-e6e11218f608 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:53.316660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491cd53d-87b2-4cbc-a40b-e6e11218f608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:53.318547 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491cd53d-87b2-4cbc-a40b-e6e11218f608 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:53.365948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1349/2699] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:53 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:53.370169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c8d55-9e4a-4c61-b8da-fda0816e43ff admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:53.380975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c8d55-9e4a-4c61-b8da-fda0816e43ff admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:53.439371 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a31cf-607d-4917-8c4d-6e16d41810c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:53.440044 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a31cf-607d-4917-8c4d-6e16d41810c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:53.440941 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a31cf-607d-4917-8c4d-6e16d41810c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:53.441220 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a31cf-607d-4917-8c4d-6e16d41810c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:53.441920 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a31cf-607d-4917-8c4d-6e16d41810c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:53.443788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a31cf-607d-4917-8c4d-6e16d41810c1 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:53.470530 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1351/2700] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:53 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:53.472280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cd53d-87b2-4cbc-a40b-e6e11218f608 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:53.483572 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cd53d-87b2-4cbc-a40b-e6e11218f608 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:53.489078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca0aec-e1a0-4c2d-93fc-9e14e4ee6348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:53.489734 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca0aec-e1a0-4c2d-93fc-9e14e4ee6348 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:53.490013 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca0aec-e1a0-4c2d-93fc-9e14e4ee6348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:53.490288 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca0aec-e1a0-4c2d-93fc-9e14e4ee6348 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:53.491069 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca0aec-e1a0-4c2d-93fc-9e14e4ee6348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:53.492645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca0aec-e1a0-4c2d-93fc-9e14e4ee6348 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:53.493985 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:53.505493 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1350/2701] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:53.509337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a31cf-607d-4917-8c4d-6e16d41810c1 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:53.519264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a31cf-607d-4917-8c4d-6e16d41810c1 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:53.523851 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e05a6-98ba-451f-9b03-283be5830bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:53.524533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e05a6-98ba-451f-9b03-283be5830bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:53.524819 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e05a6-98ba-451f-9b03-283be5830bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:53.525094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e05a6-98ba-451f-9b03-283be5830bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:53.525815 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e05a6-98ba-451f-9b03-283be5830bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:53.527385 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e05a6-98ba-451f-9b03-283be5830bd2 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:53.537898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1352/2702] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:53.541976 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca0aec-e1a0-4c2d-93fc-9e14e4ee6348 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:53.553110 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca0aec-e1a0-4c2d-93fc-9e14e4ee6348 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:53.557821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2118bd4-c0a0-429d-816e-a81afe5acaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:53.558688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2118bd4-c0a0-429d-816e-a81afe5acaf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:53.558982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2118bd4-c0a0-429d-816e-a81afe5acaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:53.559825 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2118bd4-c0a0-429d-816e-a81afe5acaf9 admin admin] PATH_INFO: `/v3/projects/ba897ed63f434ec88f93da1cae7024f1/users/5ad489ddbd3541d0b55d884bbc7f78f3/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:53.576901 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2118bd4-c0a0-429d-816e-a81afe5acaf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba897ed63f434ec88f93da1cae7024f1, user_id=5ad489ddbd3541d0b55d884bbc7f78f3, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:53.579806 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2118bd4-c0a0-429d-816e-a81afe5acaf9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:53.594188 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1351/2703] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:53 2022] PUT /identity/v3/projects/ba897ed63f434ec88f93da1cae7024f1/users/5ad489ddbd3541d0b55d884bbc7f78f3/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:53.599336 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9b5c9-0b69-4a1c-938b-af81040208e1 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:53.599838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9b5c9-0b69-4a1c-938b-af81040208e1 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:53.600005 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9b5c9-0b69-4a1c-938b-af81040208e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:53.606893 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:53.622694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-9ab9b5c9-0b69-4a1c-938b-af81040208e1 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:53.623895 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-9ab9b5c9-0b69-4a1c-938b-af81040208e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:53.682919 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1353/2704] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:53.692965 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2118bd4-c0a0-429d-816e-a81afe5acaf9 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:53.694700 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b2118bd4-c0a0-429d-816e-a81afe5acaf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:53.755865 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2118bd4-c0a0-429d-816e-a81afe5acaf9 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:53.759521 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdacc8d6-1f76-463a-92f3-42c58f0a71d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:53.760177 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacc8d6-1f76-463a-92f3-42c58f0a71d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:53.760491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacc8d6-1f76-463a-92f3-42c58f0a71d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:53.760763 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacc8d6-1f76-463a-92f3-42c58f0a71d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:53.770956 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdacc8d6-1f76-463a-92f3-42c58f0a71d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:53.772419 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdacc8d6-1f76-463a-92f3-42c58f0a71d6 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:53.783131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1352/2705] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:58.474254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b1d2d-fa01-404f-bc27-1bfd76c34596 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:58.474788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b1d2d-fa01-404f-bc27-1bfd76c34596 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:58.474788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b1d2d-fa01-404f-bc27-1bfd76c34596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:58.504374 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-2f8b1d2d-fa01-404f-bc27-1bfd76c34596 None None] MFA Rules not processed for user `5ad489ddbd3541d0b55d884bbc7f78f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:58.505912 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-2f8b1d2d-fa01-404f-bc27-1bfd76c34596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:58.577615 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1354/2706] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:30:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:58.592490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdacc8d6-1f76-463a-92f3-42c58f0a71d6 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:58.594002 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bdacc8d6-1f76-463a-92f3-42c58f0a71d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:58.677059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdacc8d6-1f76-463a-92f3-42c58f0a71d6 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:58.682833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf4e46-46e8-4c80-a46e-4fb95d078066 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:58.683573 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf4e46-46e8-4c80-a46e-4fb95d078066 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:58.685627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf4e46-46e8-4c80-a46e-4fb95d078066 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:58.685627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf4e46-46e8-4c80-a46e-4fb95d078066 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:58.694543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf4e46-46e8-4c80-a46e-4fb95d078066 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:58.696313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf4e46-46e8-4c80-a46e-4fb95d078066 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:58.706909 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1353/2707] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:31:00.275202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b108c81-3491-48dd-8844-8779eafef40d None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:00.275746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b108c81-3491-48dd-8844-8779eafef40d None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:00.275990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b108c81-3491-48dd-8844-8779eafef40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:00.295820 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-7b108c81-3491-48dd-8844-8779eafef40d None None] MFA Rules not processed for user `5ad489ddbd3541d0b55d884bbc7f78f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:00.297090 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-7b108c81-3491-48dd-8844-8779eafef40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:00.304891 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1355/2708] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:31:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:08.702606 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=72,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:08.811745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf4e46-46e8-4c80-a46e-4fb95d078066 service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:08.822991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf4e46-46e8-4c80-a46e-4fb95d078066 service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:08.826684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0617524-8b4d-447b-9fa4-be092fceff98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:08.827329 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0617524-8b4d-447b-9fa4-be092fceff98 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:08.827614 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0617524-8b4d-447b-9fa4-be092fceff98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:08.827890 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0617524-8b4d-447b-9fa4-be092fceff98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:08.830226 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c0617524-8b4d-447b-9fa4-be092fceff98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:08.894911 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0617524-8b4d-447b-9fa4-be092fceff98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:08.896376 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0617524-8b4d-447b-9fa4-be092fceff98 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:08.906082 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1354/2709] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:31:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:31:10.289193 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=61 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:11.631267 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b108c81-3491-48dd-8844-8779eafef40d None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:11.633043 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-7b108c81-3491-48dd-8844-8779eafef40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:11.648090 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b108c81-3491-48dd-8844-8779eafef40d None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:11.654504 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb4246-7adc-4f9c-84cd-85b0c4149037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:11.655388 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb4246-7adc-4f9c-84cd-85b0c4149037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:11.655658 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb4246-7adc-4f9c-84cd-85b0c4149037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:11.655931 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb4246-7adc-4f9c-84cd-85b0c4149037 admin admin] PATH_INFO: `/v3/users/a39555d792b64d23b425e1ec77ed50a0` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:11.657213 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb4246-7adc-4f9c-84cd-85b0c4149037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a39555d792b64d23b425e1ec77ed50a0)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:11.659545 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb4246-7adc-4f9c-84cd-85b0c4149037 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:11.728475 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e3eb4246-7adc-4f9c-84cd-85b0c4149037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a39555d792b64d23b425e1ec77ed50a0'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:11.742064 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e3eb4246-7adc-4f9c-84cd-85b0c4149037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a39555d792b64d23b425e1ec77ed50a0'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:11.760058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e3eb4246-7adc-4f9c-84cd-85b0c4149037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a39555d792b64d23b425e1ec77ed50a0'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:11.767796 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e3eb4246-7adc-4f9c-84cd-85b0c4149037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a39555d792b64d23b425e1ec77ed50a0'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:11.768480 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e3eb4246-7adc-4f9c-84cd-85b0c4149037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a39555d792b64d23b425e1ec77ed50a0'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:11.775456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1356/2710] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:31:11 2022] DELETE /identity/v3/users/a39555d792b64d23b425e1ec77ed50a0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:31:11.779716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9112975-fc70-4839-b38d-5317383398eb None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:11.780200 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9112975-fc70-4839-b38d-5317383398eb None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:11.780541 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9112975-fc70-4839-b38d-5317383398eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:11.799864 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f9112975-fc70-4839-b38d-5317383398eb None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:11.801160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f9112975-fc70-4839-b38d-5317383398eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:11.879613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1355/2711] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:11.890989 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb4246-7adc-4f9c-84cd-85b0c4149037 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:11.892416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e3eb4246-7adc-4f9c-84cd-85b0c4149037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:11.960364 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb4246-7adc-4f9c-84cd-85b0c4149037 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:11.965842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22599994-5bd8-46d7-b8d8-b22b7c4bf9d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:11.966512 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22599994-5bd8-46d7-b8d8-b22b7c4bf9d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:11.966797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22599994-5bd8-46d7-b8d8-b22b7c4bf9d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:11.967076 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22599994-5bd8-46d7-b8d8-b22b7c4bf9d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:11.977420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22599994-5bd8-46d7-b8d8-b22b7c4bf9d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:11.978862 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22599994-5bd8-46d7-b8d8-b22b7c4bf9d6 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:11.990416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1357/2712] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:31:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:31:12.357431 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9112975-fc70-4839-b38d-5317383398eb None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:12.359328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f9112975-fc70-4839-b38d-5317383398eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:12.372456 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9112975-fc70-4839-b38d-5317383398eb None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:12.377312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2566f-524b-47a7-aab3-50d2755fa4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:12.378046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2566f-524b-47a7-aab3-50d2755fa4b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:12.378347 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2566f-524b-47a7-aab3-50d2755fa4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:12.378674 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2566f-524b-47a7-aab3-50d2755fa4b3 admin admin] PATH_INFO: `/v3/projects/f6b182898e7b4661b59e3c5ddaec19d4` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:12.379788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2566f-524b-47a7-aab3-50d2755fa4b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6b182898e7b4661b59e3c5ddaec19d4)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:12.381738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2566f-524b-47a7-aab3-50d2755fa4b3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:12.431047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e2d2566f-524b-47a7-aab3-50d2755fa4b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6b182898e7b4661b59e3c5ddaec19d4', 'request_id': 'req-e2d2566f-524b-47a7-aab3-50d2755fa4b3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:12.439629 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e2d2566f-524b-47a7-aab3-50d2755fa4b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6b182898e7b4661b59e3c5ddaec19d4', 'request_id': 'req-e2d2566f-524b-47a7-aab3-50d2755fa4b3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:12.448633 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e2d2566f-524b-47a7-aab3-50d2755fa4b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6b182898e7b4661b59e3c5ddaec19d4', 'request_id': 'req-e2d2566f-524b-47a7-aab3-50d2755fa4b3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:12.459461 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e2d2566f-524b-47a7-aab3-50d2755fa4b3 admin admin] The token cache is being invalidate because project f6b182898e7b4661b59e3c5ddaec19d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:31:12.459750 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e2d2566f-524b-47a7-aab3-50d2755fa4b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:12.460408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e2d2566f-524b-47a7-aab3-50d2755fa4b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:12.461949 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1356/2713] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:31:12 2022] DELETE /identity/v3/projects/f6b182898e7b4661b59e3c5ddaec19d4 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:31:12.468546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f12a5-b377-47ba-81fb-445d743695be None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:12.468855 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f12a5-b377-47ba-81fb-445d743695be None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:12.469114 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f12a5-b377-47ba-81fb-445d743695be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:12.486707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-888f12a5-b377-47ba-81fb-445d743695be None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:12.487918 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-888f12a5-b377-47ba-81fb-445d743695be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:12.549108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1358/2714] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:12.557120 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df0013-ae42-4166-9f19-cd4a3a34ca15 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:12.557414 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df0013-ae42-4166-9f19-cd4a3a34ca15 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:12.557722 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df0013-ae42-4166-9f19-cd4a3a34ca15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:12.579732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-94df0013-ae42-4166-9f19-cd4a3a34ca15 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:12.580990 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-94df0013-ae42-4166-9f19-cd4a3a34ca15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:12.588963 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1357/2715] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:12.593830 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f12a5-b377-47ba-81fb-445d743695be None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:12.603873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f12a5-b377-47ba-81fb-445d743695be None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:12.608422 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b38d2-5e17-4aba-9a6c-9e6e7dde34a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:12.609073 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b38d2-5e17-4aba-9a6c-9e6e7dde34a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:12.609354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b38d2-5e17-4aba-9a6c-9e6e7dde34a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:12.609818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b38d2-5e17-4aba-9a6c-9e6e7dde34a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:12.610554 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b38d2-5e17-4aba-9a6c-9e6e7dde34a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:31:12.610859 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b38d2-5e17-4aba-9a6c-9e6e7dde34a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:12.612447 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b38d2-5e17-4aba-9a6c-9e6e7dde34a7 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:12.623992 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1359/2716] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:31:12.628424 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df0013-ae42-4166-9f19-cd4a3a34ca15 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:12.638339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df0013-ae42-4166-9f19-cd4a3a34ca15 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:12.643248 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b648e1-e6f8-4b79-a0b4-48fdf27b2644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:12.643892 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b648e1-e6f8-4b79-a0b4-48fdf27b2644 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:12.644172 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b648e1-e6f8-4b79-a0b4-48fdf27b2644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:12.644536 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b648e1-e6f8-4b79-a0b4-48fdf27b2644 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:12.645531 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b648e1-e6f8-4b79-a0b4-48fdf27b2644 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:12.647761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b648e1-e6f8-4b79-a0b4-48fdf27b2644 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:12.658554 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:12.677189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1358/2717] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:31:12 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:31:12.681438 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b38d2-5e17-4aba-9a6c-9e6e7dde34a7 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:12.686979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:12.694884 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b38d2-5e17-4aba-9a6c-9e6e7dde34a7 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:12.763050 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e1858-aaea-4b9b-89ca-063a12adc4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:12.763877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e1858-aaea-4b9b-89ca-063a12adc4ec admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:12.764160 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e1858-aaea-4b9b-89ca-063a12adc4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:12.768127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e1858-aaea-4b9b-89ca-063a12adc4ec admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:12.768127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e1858-aaea-4b9b-89ca-063a12adc4ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:12.768127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e1858-aaea-4b9b-89ca-063a12adc4ec admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:12.796555 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1360/2718] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:31:12 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:31:12.800767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b648e1-e6f8-4b79-a0b4-48fdf27b2644 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:12.812768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b648e1-e6f8-4b79-a0b4-48fdf27b2644 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:12.818620 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a771273-9143-4467-a06a-e326d3589ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:12.819376 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a771273-9143-4467-a06a-e326d3589ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:12.819853 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a771273-9143-4467-a06a-e326d3589ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:12.820408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a771273-9143-4467-a06a-e326d3589ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:12.821352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a771273-9143-4467-a06a-e326d3589ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:12.823040 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a771273-9143-4467-a06a-e326d3589ff7 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:12.838354 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1359/2719] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:31:12.839683 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e1858-aaea-4b9b-89ca-063a12adc4ec admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:12.850036 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e1858-aaea-4b9b-89ca-063a12adc4ec admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:12.854805 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e10bf-a80d-4434-bb0c-60715ae4c7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:12.855581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e10bf-a80d-4434-bb0c-60715ae4c7bd admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:12.856003 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e10bf-a80d-4434-bb0c-60715ae4c7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:12.856433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e10bf-a80d-4434-bb0c-60715ae4c7bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:12.857345 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e10bf-a80d-4434-bb0c-60715ae4c7bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:12.859088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e10bf-a80d-4434-bb0c-60715ae4c7bd admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:12.869529 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1361/2720] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:31:12.874160 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a771273-9143-4467-a06a-e326d3589ff7 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:12.884199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a771273-9143-4467-a06a-e326d3589ff7 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:12.889642 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c4862-78ff-4d43-a20b-1b455408c71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:12.890521 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c4862-78ff-4d43-a20b-1b455408c71e admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:12.890815 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c4862-78ff-4d43-a20b-1b455408c71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:12.891204 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c4862-78ff-4d43-a20b-1b455408c71e admin admin] PATH_INFO: `/v3/projects/4e5d98e3bb2a4708a934a15674033c12/users/e1e97b329ce34e8ca7ca7d3e046a3136/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:12.907316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c4862-78ff-4d43-a20b-1b455408c71e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e5d98e3bb2a4708a934a15674033c12, user_id=e1e97b329ce34e8ca7ca7d3e046a3136, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:12.910433 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c4862-78ff-4d43-a20b-1b455408c71e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:12.925063 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1360/2721] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:31:12 2022] PUT /identity/v3/projects/4e5d98e3bb2a4708a934a15674033c12/users/e1e97b329ce34e8ca7ca7d3e046a3136/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:31:12.930189 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a49df-ac1d-438f-8292-9a8d4af1adcc None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:12.930581 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a49df-ac1d-438f-8292-9a8d4af1adcc None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:12.930963 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a49df-ac1d-438f-8292-9a8d4af1adcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:12.950507 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-875a49df-ac1d-438f-8292-9a8d4af1adcc None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:12.951718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-875a49df-ac1d-438f-8292-9a8d4af1adcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:13.026307 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1362/2722] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:13.037462 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c4862-78ff-4d43-a20b-1b455408c71e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:13.040483 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-624c4862-78ff-4d43-a20b-1b455408c71e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:13.098782 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c4862-78ff-4d43-a20b-1b455408c71e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:13.102448 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f91afb-4929-4063-ac5d-efad08b50297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:13.103100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f91afb-4929-4063-ac5d-efad08b50297 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:13.103383 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f91afb-4929-4063-ac5d-efad08b50297 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:13.103660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f91afb-4929-4063-ac5d-efad08b50297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:13.113352 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f91afb-4929-4063-ac5d-efad08b50297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:13.114904 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f91afb-4929-4063-ac5d-efad08b50297 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:13.125092 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1361/2723] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:31:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:31:18.675887 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aec076-a38f-4870-a92d-50fbfdd87020 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:18.676358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aec076-a38f-4870-a92d-50fbfdd87020 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:18.676447 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aec076-a38f-4870-a92d-50fbfdd87020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:18.707869 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-91aec076-a38f-4870-a92d-50fbfdd87020 None None] MFA Rules not processed for user `e1e97b329ce34e8ca7ca7d3e046a3136`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:18.709384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-91aec076-a38f-4870-a92d-50fbfdd87020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:18.783821 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1363/2724] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:31:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:18.788077 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f91afb-4929-4063-ac5d-efad08b50297 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:18.798448 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f91afb-4929-4063-ac5d-efad08b50297 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:18.803981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00d200-675e-4912-826d-00979f09e138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:18.804718 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00d200-675e-4912-826d-00979f09e138 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:18.805007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00d200-675e-4912-826d-00979f09e138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:18.805282 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00d200-675e-4912-826d-00979f09e138 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:18.805987 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee00d200-675e-4912-826d-00979f09e138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:18.807945 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee00d200-675e-4912-826d-00979f09e138 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:18.839509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1362/2725] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:31:18 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:31:18.841194 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aec076-a38f-4870-a92d-50fbfdd87020 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:18.852538 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aec076-a38f-4870-a92d-50fbfdd87020 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:18.861025 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:18.917358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ffb8f-2fa2-4d80-8731-042b5e8bab5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:18.918048 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ffb8f-2fa2-4d80-8731-042b5e8bab5f admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:18.918329 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ffb8f-2fa2-4d80-8731-042b5e8bab5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:18.918600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ffb8f-2fa2-4d80-8731-042b5e8bab5f admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:18.919296 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ffb8f-2fa2-4d80-8731-042b5e8bab5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:18.921306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ffb8f-2fa2-4d80-8731-042b5e8bab5f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:18.943728 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1364/2726] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:31:18 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:31:18.947783 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00d200-675e-4912-826d-00979f09e138 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:18.959566 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00d200-675e-4912-826d-00979f09e138 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:18.964542 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c84c05-2700-427a-8ea7-8fe61a241e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:18.965250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c84c05-2700-427a-8ea7-8fe61a241e75 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:18.965531 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c84c05-2700-427a-8ea7-8fe61a241e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:18.965805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c84c05-2700-427a-8ea7-8fe61a241e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:18.966556 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c84c05-2700-427a-8ea7-8fe61a241e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:18.968131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c84c05-2700-427a-8ea7-8fe61a241e75 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:18.982421 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1363/2727] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:31:18.983562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ffb8f-2fa2-4d80-8731-042b5e8bab5f admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:18.995477 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ffb8f-2fa2-4d80-8731-042b5e8bab5f admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:19.001501 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0806003-aa78-4bab-811c-ead5eb24220a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:19.002340 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0806003-aa78-4bab-811c-ead5eb24220a admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:19.002671 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0806003-aa78-4bab-811c-ead5eb24220a admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:19.002988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0806003-aa78-4bab-811c-ead5eb24220a admin admin] PATH_INFO: `/v3/projects/35acb2515a4d449a85f918e18041d387/users/4be3bb2a783443988e8f1a07f9daecf9/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:19.020905 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0806003-aa78-4bab-811c-ead5eb24220a admin admin] RBAC: Authorizing `identity:create_grant(project_id=35acb2515a4d449a85f918e18041d387, user_id=4be3bb2a783443988e8f1a07f9daecf9, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:19.023533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0806003-aa78-4bab-811c-ead5eb24220a admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:19.036992 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1365/2728] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:31:18 2022] PUT /identity/v3/projects/35acb2515a4d449a85f918e18041d387/users/4be3bb2a783443988e8f1a07f9daecf9/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:31:19.040712 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c84c05-2700-427a-8ea7-8fe61a241e75 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:19.052185 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c84c05-2700-427a-8ea7-8fe61a241e75 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:19.112914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a8f1c-7e29-491e-a985-2b0b69327561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:19.113519 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a8f1c-7e29-491e-a985-2b0b69327561 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:19.113802 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a8f1c-7e29-491e-a985-2b0b69327561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:19.114091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a8f1c-7e29-491e-a985-2b0b69327561 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:19.114807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a8f1c-7e29-491e-a985-2b0b69327561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:19.116462 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a8f1c-7e29-491e-a985-2b0b69327561 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:19.127803 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1364/2729] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:31:19.131982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0806003-aa78-4bab-811c-ead5eb24220a admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:19.142965 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0806003-aa78-4bab-811c-ead5eb24220a admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:19.147731 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8d9ce-141f-4f2c-af31-dcff9f93a210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:19.148667 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8d9ce-141f-4f2c-af31-dcff9f93a210 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:19.148953 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8d9ce-141f-4f2c-af31-dcff9f93a210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:19.149227 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8d9ce-141f-4f2c-af31-dcff9f93a210 admin admin] PATH_INFO: `/v3/projects/35acb2515a4d449a85f918e18041d387/users/4be3bb2a783443988e8f1a07f9daecf9/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:19.151238 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8d9ce-141f-4f2c-af31-dcff9f93a210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35acb2515a4d449a85f918e18041d387, user_id=4be3bb2a783443988e8f1a07f9daecf9, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:19.153946 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8d9ce-141f-4f2c-af31-dcff9f93a210 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:19.170228 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1366/2730] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:31:19 2022] PUT /identity/v3/projects/35acb2515a4d449a85f918e18041d387/users/4be3bb2a783443988e8f1a07f9daecf9/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:31:23.514306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb7d8e-e31e-4a8c-b3dc-71fe8cf77c1b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:23.514853 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb7d8e-e31e-4a8c-b3dc-71fe8cf77c1b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:23.514984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb7d8e-e31e-4a8c-b3dc-71fe8cf77c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:23.545510 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d9cb7d8e-e31e-4a8c-b3dc-71fe8cf77c1b None None] MFA Rules not processed for user `4be3bb2a783443988e8f1a07f9daecf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:23.546795 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d9cb7d8e-e31e-4a8c-b3dc-71fe8cf77c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:23.555738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:23.625935 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1365/2731] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:31:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:23.641685 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8d9ce-141f-4f2c-af31-dcff9f93a210 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:23.643439 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4ba8d9ce-141f-4f2c-af31-dcff9f93a210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:23.721360 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8d9ce-141f-4f2c-af31-dcff9f93a210 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:23.727159 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3bf1b-56c3-4fae-9a8c-c1628a5470e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:23.727893 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3bf1b-56c3-4fae-9a8c-c1628a5470e9 service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:23.728257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3bf1b-56c3-4fae-9a8c-c1628a5470e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:23.728555 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3bf1b-56c3-4fae-9a8c-c1628a5470e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:23.787632 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3bf1b-56c3-4fae-9a8c-c1628a5470e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:23.789259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3bf1b-56c3-4fae-9a8c-c1628a5470e9 service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:23.795434 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:23.802077 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1367/2732] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:31:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:31:24.935559 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0436aa3-17a4-4919-ac07-f13b79bcd243 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:24.936076 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0436aa3-17a4-4919-ac07-f13b79bcd243 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:24.936176 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0436aa3-17a4-4919-ac07-f13b79bcd243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:24.967259 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f0436aa3-17a4-4919-ac07-f13b79bcd243 None None] MFA Rules not processed for user `e1e97b329ce34e8ca7ca7d3e046a3136`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:24.968532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f0436aa3-17a4-4919-ac07-f13b79bcd243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:24.976175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1366/2733] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:31:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:34.961058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=28 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:32:05.453160 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb12d66-500c-4cf5-a8a4-f1e9c0167405 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:05.453750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb12d66-500c-4cf5-a8a4-f1e9c0167405 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:05.454121 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb12d66-500c-4cf5-a8a4-f1e9c0167405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:05.482671 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-bfb12d66-500c-4cf5-a8a4-f1e9c0167405 None None] MFA Rules not processed for user `44d26a97291e44c78d3a6438d7ab76c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:32:05.483864 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-bfb12d66-500c-4cf5-a8a4-f1e9c0167405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:05.555709 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1368/2734] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:32:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:32:06.662157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0436aa3-17a4-4919-ac07-f13b79bcd243 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:06.665182 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f0436aa3-17a4-4919-ac07-f13b79bcd243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:06.784327 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0436aa3-17a4-4919-ac07-f13b79bcd243 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:06.788142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01cdf61-483a-4096-bc5a-2c03f7f16ab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:06.788823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cdf61-483a-4096-bc5a-2c03f7f16ab8 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:06.789142 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cdf61-483a-4096-bc5a-2c03f7f16ab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:06.789381 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cdf61-483a-4096-bc5a-2c03f7f16ab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:06.808217 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01cdf61-483a-4096-bc5a-2c03f7f16ab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:06.809757 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01cdf61-483a-4096-bc5a-2c03f7f16ab8 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:06.821439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1367/2735] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:32:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:32:06.927224 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb12d66-500c-4cf5-a8a4-f1e9c0167405 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:06.929180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-bfb12d66-500c-4cf5-a8a4-f1e9c0167405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:07.035087 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb12d66-500c-4cf5-a8a4-f1e9c0167405 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:07.040233 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1676c-8bab-4966-a6dd-3990a0f849cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae581d3dda354e07894cb5205d1b2c5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:07.040923 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1676c-8bab-4966-a6dd-3990a0f849cd service placement] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:07.041208 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1676c-8bab-4966-a6dd-3990a0f849cd service placement] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:07.041481 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1676c-8bab-4966-a6dd-3990a0f849cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:07.043307 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e9e1676c-8bab-4966-a6dd-3990a0f849cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:07.056603 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1676c-8bab-4966-a6dd-3990a0f849cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:07.058217 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1676c-8bab-4966-a6dd-3990a0f849cd service placement] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:07.069707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1369/2736] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:32:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:32:14.181028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793604c4-04c0-4cca-bb78-7bde513179a9 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:14.181554 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793604c4-04c0-4cca-bb78-7bde513179a9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:14.181664 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793604c4-04c0-4cca-bb78-7bde513179a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:14.201128 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-793604c4-04c0-4cca-bb78-7bde513179a9 None None] MFA Rules not processed for user `5ad489ddbd3541d0b55d884bbc7f78f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:32:14.202371 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-793604c4-04c0-4cca-bb78-7bde513179a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:14.257496 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1368/2737] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:32:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:32:17.065434 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=78 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:32:24.250321 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=76 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:07.914007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1676c-8bab-4966-a6dd-3990a0f849cd service placement] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:07.915570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e9e1676c-8bab-4966-a6dd-3990a0f849cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:07.929235 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1676c-8bab-4966-a6dd-3990a0f849cd service placement] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:07.934186 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fabbf-535e-4262-a201-873ceffd18a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae581d3dda354e07894cb5205d1b2c5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:07.934827 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fabbf-535e-4262-a201-873ceffd18a4 service placement] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:07.935109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fabbf-535e-4262-a201-873ceffd18a4 service placement] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:07.935527 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fabbf-535e-4262-a201-873ceffd18a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:07.937132 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-a55fabbf-535e-4262-a201-873ceffd18a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:07.950139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fabbf-535e-4262-a201-873ceffd18a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:07.951647 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fabbf-535e-4262-a201-873ceffd18a4 service placement] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:07.962230 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1370/2738] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:33:07.994652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793604c4-04c0-4cca-bb78-7bde513179a9 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:07.996122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-793604c4-04c0-4cca-bb78-7bde513179a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:08.104173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793604c4-04c0-4cca-bb78-7bde513179a9 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:08.108967 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bfb07-972b-4daa-8f5d-44fa276c691b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:08.109756 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bfb07-972b-4daa-8f5d-44fa276c691b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:08.110038 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bfb07-972b-4daa-8f5d-44fa276c691b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:08.110316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bfb07-972b-4daa-8f5d-44fa276c691b admin admin] PATH_INFO: `/v3/users/44d26a97291e44c78d3a6438d7ab76c5` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:08.111560 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6bfb07-972b-4daa-8f5d-44fa276c691b admin admin] RBAC: Authorizing `identity:delete_user(user_id=44d26a97291e44c78d3a6438d7ab76c5)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:08.114189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6bfb07-972b-4daa-8f5d-44fa276c691b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:08.193033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7e6bfb07-972b-4daa-8f5d-44fa276c691b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44d26a97291e44c78d3a6438d7ab76c5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:08.203846 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7e6bfb07-972b-4daa-8f5d-44fa276c691b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44d26a97291e44c78d3a6438d7ab76c5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:08.205500 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7e6bfb07-972b-4daa-8f5d-44fa276c691b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44d26a97291e44c78d3a6438d7ab76c5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:08.218190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7e6bfb07-972b-4daa-8f5d-44fa276c691b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44d26a97291e44c78d3a6438d7ab76c5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:08.235058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7e6bfb07-972b-4daa-8f5d-44fa276c691b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44d26a97291e44c78d3a6438d7ab76c5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:08.249371 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1369/2739] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:33:07 2022] DELETE /identity/v3/users/44d26a97291e44c78d3a6438d7ab76c5 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:33:08.726986 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fabbf-535e-4262-a201-873ceffd18a4 service placement] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:08.739380 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fabbf-535e-4262-a201-873ceffd18a4 service placement] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:08.816566 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256ab05-b497-4be8-a0b7-9acab88e164d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:08.817442 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256ab05-b497-4be8-a0b7-9acab88e164d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:08.817730 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256ab05-b497-4be8-a0b7-9acab88e164d admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:08.818012 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256ab05-b497-4be8-a0b7-9acab88e164d admin admin] PATH_INFO: `/v3/projects/7ee2fc4ab6a540cc852d2b3850e6b550` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:08.819209 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256ab05-b497-4be8-a0b7-9acab88e164d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ee2fc4ab6a540cc852d2b3850e6b550)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:08.821171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256ab05-b497-4be8-a0b7-9acab88e164d admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:08.868959 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-1256ab05-b497-4be8-a0b7-9acab88e164d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ee2fc4ab6a540cc852d2b3850e6b550', 'request_id': 'req-1256ab05-b497-4be8-a0b7-9acab88e164d'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:08.878003 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-1256ab05-b497-4be8-a0b7-9acab88e164d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ee2fc4ab6a540cc852d2b3850e6b550', 'request_id': 'req-1256ab05-b497-4be8-a0b7-9acab88e164d'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:08.884465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-1256ab05-b497-4be8-a0b7-9acab88e164d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ee2fc4ab6a540cc852d2b3850e6b550', 'request_id': 'req-1256ab05-b497-4be8-a0b7-9acab88e164d'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:08.898338 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-1256ab05-b497-4be8-a0b7-9acab88e164d admin admin] The token cache is being invalidate because project 7ee2fc4ab6a540cc852d2b3850e6b550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:33:08.898628 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-1256ab05-b497-4be8-a0b7-9acab88e164d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:08.899290 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-1256ab05-b497-4be8-a0b7-9acab88e164d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:08.900886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1371/2740] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:33:08 2022] DELETE /identity/v3/projects/7ee2fc4ab6a540cc852d2b3850e6b550 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:33:18.239628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=66,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:18.896193 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:19.167698 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bfb07-972b-4daa-8f5d-44fa276c691b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:19.169405 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7e6bfb07-972b-4daa-8f5d-44fa276c691b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:19.233311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bfb07-972b-4daa-8f5d-44fa276c691b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:19.296863 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb215b9c-07ae-4f8f-b5cd-ed318c567dcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:19.297906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb215b9c-07ae-4f8f-b5cd-ed318c567dcb service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:19.298306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb215b9c-07ae-4f8f-b5cd-ed318c567dcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:19.298741 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb215b9c-07ae-4f8f-b5cd-ed318c567dcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:19.300759 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bb215b9c-07ae-4f8f-b5cd-ed318c567dcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:19.374314 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb215b9c-07ae-4f8f-b5cd-ed318c567dcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:19.375838 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb215b9c-07ae-4f8f-b5cd-ed318c567dcb service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:19.387248 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1370/2741] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:33:26.384255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256ab05-b497-4be8-a0b7-9acab88e164d admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:26.385772 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-1256ab05-b497-4be8-a0b7-9acab88e164d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:26.400832 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256ab05-b497-4be8-a0b7-9acab88e164d admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:26.406133 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca9d8f-1f74-4ef6-a76b-a82b2e9594ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:26.407083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca9d8f-1f74-4ef6-a76b-a82b2e9594ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:26.407509 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca9d8f-1f74-4ef6-a76b-a82b2e9594ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:26.407878 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca9d8f-1f74-4ef6-a76b-a82b2e9594ff admin admin] PATH_INFO: `/v3/users/4a4f4d585adf40b880be44c29ecd53cd` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:26.414550 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca9d8f-1f74-4ef6-a76b-a82b2e9594ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a4f4d585adf40b880be44c29ecd53cd)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:26.414550 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca9d8f-1f74-4ef6-a76b-a82b2e9594ff admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:26.466237 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-13ca9d8f-1f74-4ef6-a76b-a82b2e9594ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a4f4d585adf40b880be44c29ecd53cd'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.474181 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-13ca9d8f-1f74-4ef6-a76b-a82b2e9594ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a4f4d585adf40b880be44c29ecd53cd'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.486517 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-13ca9d8f-1f74-4ef6-a76b-a82b2e9594ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a4f4d585adf40b880be44c29ecd53cd'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.493259 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-13ca9d8f-1f74-4ef6-a76b-a82b2e9594ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a4f4d585adf40b880be44c29ecd53cd'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.493957 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-13ca9d8f-1f74-4ef6-a76b-a82b2e9594ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a4f4d585adf40b880be44c29ecd53cd'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.500207 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1372/2742] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:33:26 2022] DELETE /identity/v3/users/4a4f4d585adf40b880be44c29ecd53cd => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:33:26.677354 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb215b9c-07ae-4f8f-b5cd-ed318c567dcb service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:26.697243 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb215b9c-07ae-4f8f-b5cd-ed318c567dcb service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:26.761816 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:26.762848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:26.763135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:26.763410 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072 admin admin] PATH_INFO: `/v3/projects/182d5b6ac9e6412b907119a623869c80` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:26.764679 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=182d5b6ac9e6412b907119a623869c80)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:26.766574 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:26.774839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:26.816628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '182d5b6ac9e6412b907119a623869c80', 'request_id': 'req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.822891 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '182d5b6ac9e6412b907119a623869c80', 'request_id': 'req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.830470 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '182d5b6ac9e6412b907119a623869c80', 'request_id': 'req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.832220 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e8e02-7889-4124-bcc9-b514966a67bb None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:26.832551 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e8e02-7889-4124-bcc9-b514966a67bb None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:26.832829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e8e02-7889-4124-bcc9-b514966a67bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:26.841768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072 admin admin] The token cache is being invalidate because project 182d5b6ac9e6412b907119a623869c80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:33:26.842061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.842709 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.844272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1371/2743] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:33:26 2022] DELETE /identity/v3/projects/182d5b6ac9e6412b907119a623869c80 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:33:26.848056 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:26.849318 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:26.850615 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-ad4e8e02-7889-4124-bcc9-b514966a67bb None None] MFA Rules not processed for user `5ad489ddbd3541d0b55d884bbc7f78f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:33:26.851791 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ad4e8e02-7889-4124-bcc9-b514966a67bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:26.907150 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1373/2744] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:33:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:33:26.915328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26e4a5-ebcc-42f2-a809-96ff6e15c072 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:26.920538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5727734-4dc2-4804-84c0-23a473201b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:26.921532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5727734-4dc2-4804-84c0-23a473201b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:26.921810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5727734-4dc2-4804-84c0-23a473201b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:26.922080 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5727734-4dc2-4804-84c0-23a473201b1b admin admin] PATH_INFO: `/v3/users/413e322998344953a89da5bbbc12fabe` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:26.923877 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5727734-4dc2-4804-84c0-23a473201b1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=413e322998344953a89da5bbbc12fabe)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:26.925901 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5727734-4dc2-4804-84c0-23a473201b1b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:26.982251 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b5727734-4dc2-4804-84c0-23a473201b1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '413e322998344953a89da5bbbc12fabe'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.990316 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b5727734-4dc2-4804-84c0-23a473201b1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '413e322998344953a89da5bbbc12fabe'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.991000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b5727734-4dc2-4804-84c0-23a473201b1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '413e322998344953a89da5bbbc12fabe'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.998306 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b5727734-4dc2-4804-84c0-23a473201b1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '413e322998344953a89da5bbbc12fabe'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:27.011300 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b5727734-4dc2-4804-84c0-23a473201b1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '413e322998344953a89da5bbbc12fabe'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:27.017408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1372/2745] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:33:26 2022] DELETE /identity/v3/users/413e322998344953a89da5bbbc12fabe => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:33:27.329046 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e8e02-7889-4124-bcc9-b514966a67bb None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:27.340889 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e8e02-7889-4124-bcc9-b514966a67bb None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:27.407110 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2211f8-ad9e-45fc-b8dc-2c591533020b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:27.408782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2211f8-ad9e-45fc-b8dc-2c591533020b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:27.409273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2211f8-ad9e-45fc-b8dc-2c591533020b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:27.409780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2211f8-ad9e-45fc-b8dc-2c591533020b admin admin] PATH_INFO: `/v3/projects/4e6e524426f1439c99af0938f0e5a6ab` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:27.411807 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2211f8-ad9e-45fc-b8dc-2c591533020b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e6e524426f1439c99af0938f0e5a6ab)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:27.413792 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2211f8-ad9e-45fc-b8dc-2c591533020b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:27.440333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:27.473970 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-8c2211f8-ad9e-45fc-b8dc-2c591533020b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e6e524426f1439c99af0938f0e5a6ab', 'request_id': 'req-8c2211f8-ad9e-45fc-b8dc-2c591533020b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:27.482257 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-8c2211f8-ad9e-45fc-b8dc-2c591533020b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e6e524426f1439c99af0938f0e5a6ab', 'request_id': 'req-8c2211f8-ad9e-45fc-b8dc-2c591533020b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:27.488609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-8c2211f8-ad9e-45fc-b8dc-2c591533020b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e6e524426f1439c99af0938f0e5a6ab', 'request_id': 'req-8c2211f8-ad9e-45fc-b8dc-2c591533020b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:27.501412 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-8c2211f8-ad9e-45fc-b8dc-2c591533020b admin admin] The token cache is being invalidate because project 4e6e524426f1439c99af0938f0e5a6ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:33:27.502465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-8c2211f8-ad9e-45fc-b8dc-2c591533020b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:27.503160 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-8c2211f8-ad9e-45fc-b8dc-2c591533020b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:27.505333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1374/2746] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:33:27 2022] DELETE /identity/v3/projects/4e6e524426f1439c99af0938f0e5a6ab => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:33:37.013809 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=74,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:37.497981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:34:17.048807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5727734-4dc2-4804-84c0-23a473201b1b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:34:17.050214 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b5727734-4dc2-4804-84c0-23a473201b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:17.127493 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5727734-4dc2-4804-84c0-23a473201b1b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:34:17.174788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa129d-a9f2-4d65-9503-3bf574760c7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:34:17.175426 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa129d-a9f2-4d65-9503-3bf574760c7a service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:17.175708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa129d-a9f2-4d65-9503-3bf574760c7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:17.175982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa129d-a9f2-4d65-9503-3bf574760c7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:17.177774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-73fa129d-a9f2-4d65-9503-3bf574760c7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:17.263552 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa129d-a9f2-4d65-9503-3bf574760c7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:34:17.264931 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa129d-a9f2-4d65-9503-3bf574760c7a service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:34:17.285188 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1373/2747] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:34:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:34:22.071743 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2211f8-ad9e-45fc-b8dc-2c591533020b admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:34:22.073189 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-8c2211f8-ad9e-45fc-b8dc-2c591533020b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:22.131143 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2211f8-ad9e-45fc-b8dc-2c591533020b admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:34:22.134750 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a143c-a80b-4932-89b5-f7be3df35f39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:34:22.135402 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a143c-a80b-4932-89b5-f7be3df35f39 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:22.135687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a143c-a80b-4932-89b5-f7be3df35f39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:22.136004 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a143c-a80b-4932-89b5-f7be3df35f39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:22.137788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4d9a143c-a80b-4932-89b5-f7be3df35f39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:22.151727 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a143c-a80b-4932-89b5-f7be3df35f39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:34:22.153168 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a143c-a80b-4932-89b5-f7be3df35f39 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:34:22.164125 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1375/2748] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:34:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:34:27.282640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:34:32.001012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d52676-9dcb-43c2-bf41-2e7b5037de5f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:32.001648 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d52676-9dcb-43c2-bf41-2e7b5037de5f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:32.001648 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d52676-9dcb-43c2-bf41-2e7b5037de5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:32.018950 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-65d52676-9dcb-43c2-bf41-2e7b5037de5f None None] MFA Rules not processed for user `5ad489ddbd3541d0b55d884bbc7f78f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:34:32.020134 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-65d52676-9dcb-43c2-bf41-2e7b5037de5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:32.082590 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1374/2749] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:34:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:34:32.159152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=26 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:34:35.496665 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a143c-a80b-4932-89b5-f7be3df35f39 service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:34:35.498111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4d9a143c-a80b-4932-89b5-f7be3df35f39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:35.509846 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a143c-a80b-4932-89b5-f7be3df35f39 service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:34:35.515596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1323b-4ce1-4581-89ca-edf9d279306b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:34:35.516260 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1323b-4ce1-4581-89ca-edf9d279306b service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:35.516577 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1323b-4ce1-4581-89ca-edf9d279306b service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:35.516854 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1323b-4ce1-4581-89ca-edf9d279306b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:35.518687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-ffb1323b-4ce1-4581-89ca-edf9d279306b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:35.529646 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb1323b-4ce1-4581-89ca-edf9d279306b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:34:35.531105 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb1323b-4ce1-4581-89ca-edf9d279306b service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:34:35.540052 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1376/2750] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:34:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:34:35.668660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d52676-9dcb-43c2-bf41-2e7b5037de5f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:34:35.674456 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1323b-4ce1-4581-89ca-edf9d279306b service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:34:35.678865 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d52676-9dcb-43c2-bf41-2e7b5037de5f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:34:35.682741 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfd8d7-a67c-4039-8f52-fecd69cff920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:34:35.683487 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfd8d7-a67c-4039-8f52-fecd69cff920 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:35.683772 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfd8d7-a67c-4039-8f52-fecd69cff920 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:35.684046 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfd8d7-a67c-4039-8f52-fecd69cff920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:35.684161 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1323b-4ce1-4581-89ca-edf9d279306b service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:34:35.685873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cbcfd8d7-a67c-4039-8f52-fecd69cff920 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:35.688426 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4629ac7-43b3-43c0-bd01-61408f7f274b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:34:35.689274 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4629ac7-43b3-43c0-bd01-61408f7f274b service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:35.689634 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4629ac7-43b3-43c0-bd01-61408f7f274b service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:35.689851 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4629ac7-43b3-43c0-bd01-61408f7f274b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:35.691707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-b4629ac7-43b3-43c0-bd01-61408f7f274b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:35.700954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfd8d7-a67c-4039-8f52-fecd69cff920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:34:35.702388 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfd8d7-a67c-4039-8f52-fecd69cff920 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:34:35.702863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4629ac7-43b3-43c0-bd01-61408f7f274b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:34:35.713812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4629ac7-43b3-43c0-bd01-61408f7f274b service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:34:35.713812 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1377/2751] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:34:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:34:35.717622 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1375/2752] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:34:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:34:45.710475 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=12 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:34:45.711038 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=31 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:44.900205 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4629ac7-43b3-43c0-bd01-61408f7f274b service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:44.901684 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-b4629ac7-43b3-43c0-bd01-61408f7f274b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:44.951693 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfd8d7-a67c-4039-8f52-fecd69cff920 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:44.953745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cbcfd8d7-a67c-4039-8f52-fecd69cff920 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:44.986740 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4629ac7-43b3-43c0-bd01-61408f7f274b service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:44.992679 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72f029-2ae0-4268-8719-e1eac4e03098 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae581d3dda354e07894cb5205d1b2c5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:44.993406 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72f029-2ae0-4268-8719-e1eac4e03098 service placement] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:44.993763 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72f029-2ae0-4268-8719-e1eac4e03098 service placement] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:44.994113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72f029-2ae0-4268-8719-e1eac4e03098 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:44.996525 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-9b72f029-2ae0-4268-8719-e1eac4e03098 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:45.009623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b72f029-2ae0-4268-8719-e1eac4e03098 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:45.011263 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b72f029-2ae0-4268-8719-e1eac4e03098 service placement] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:45.021250 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1378/2753] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:35:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:35:45.024859 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfd8d7-a67c-4039-8f52-fecd69cff920 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:45.029639 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6adaf8-120f-42f3-bf6e-c937041ba575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:45.030543 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6adaf8-120f-42f3-bf6e-c937041ba575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.030824 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6adaf8-120f-42f3-bf6e-c937041ba575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.031098 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6adaf8-120f-42f3-bf6e-c937041ba575 admin admin] PATH_INFO: `/v3/users/5ad489ddbd3541d0b55d884bbc7f78f3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.032307 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6adaf8-120f-42f3-bf6e-c937041ba575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ad489ddbd3541d0b55d884bbc7f78f3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:45.034428 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6adaf8-120f-42f3-bf6e-c937041ba575 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:45.086626 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ab6adaf8-120f-42f3-bf6e-c937041ba575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ad489ddbd3541d0b55d884bbc7f78f3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:45.094064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ab6adaf8-120f-42f3-bf6e-c937041ba575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ad489ddbd3541d0b55d884bbc7f78f3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:45.094704 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ab6adaf8-120f-42f3-bf6e-c937041ba575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ad489ddbd3541d0b55d884bbc7f78f3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:45.102437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ab6adaf8-120f-42f3-bf6e-c937041ba575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ad489ddbd3541d0b55d884bbc7f78f3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:45.115603 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ab6adaf8-120f-42f3-bf6e-c937041ba575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ad489ddbd3541d0b55d884bbc7f78f3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:45.122302 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1376/2754] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:35:44 2022] DELETE /identity/v3/users/5ad489ddbd3541d0b55d884bbc7f78f3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:45.312310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72f029-2ae0-4268-8719-e1eac4e03098 service placement] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:45.322782 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72f029-2ae0-4268-8719-e1eac4e03098 service placement] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:45.388624 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62caa10-6dcb-4d60-b2eb-8f35f0f17e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:45.389481 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62caa10-6dcb-4d60-b2eb-8f35f0f17e7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.389756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62caa10-6dcb-4d60-b2eb-8f35f0f17e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.390035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62caa10-6dcb-4d60-b2eb-8f35f0f17e7e admin admin] PATH_INFO: `/v3/projects/ba897ed63f434ec88f93da1cae7024f1` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.391205 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62caa10-6dcb-4d60-b2eb-8f35f0f17e7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba897ed63f434ec88f93da1cae7024f1)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:45.393115 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62caa10-6dcb-4d60-b2eb-8f35f0f17e7e admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:45.440162 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-f62caa10-6dcb-4d60-b2eb-8f35f0f17e7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba897ed63f434ec88f93da1cae7024f1', 'request_id': 'req-f62caa10-6dcb-4d60-b2eb-8f35f0f17e7e'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:45.455334 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-f62caa10-6dcb-4d60-b2eb-8f35f0f17e7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba897ed63f434ec88f93da1cae7024f1', 'request_id': 'req-f62caa10-6dcb-4d60-b2eb-8f35f0f17e7e'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:45.461790 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-f62caa10-6dcb-4d60-b2eb-8f35f0f17e7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba897ed63f434ec88f93da1cae7024f1', 'request_id': 'req-f62caa10-6dcb-4d60-b2eb-8f35f0f17e7e'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:45.473119 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-f62caa10-6dcb-4d60-b2eb-8f35f0f17e7e admin admin] The token cache is being invalidate because project ba897ed63f434ec88f93da1cae7024f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:35:45.473404 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-f62caa10-6dcb-4d60-b2eb-8f35f0f17e7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:45.474595 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-f62caa10-6dcb-4d60-b2eb-8f35f0f17e7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:45.476142 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1379/2755] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:35:45 2022] DELETE /identity/v3/projects/ba897ed63f434ec88f93da1cae7024f1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:45.482270 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33ab37-b91e-4dbc-99a9-9245768a0d2f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.482914 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33ab37-b91e-4dbc-99a9-9245768a0d2f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.483417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33ab37-b91e-4dbc-99a9-9245768a0d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.565904 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ee33ab37-b91e-4dbc-99a9-9245768a0d2f None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:45.567123 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ee33ab37-b91e-4dbc-99a9-9245768a0d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:45.612141 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:45.629814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1377/2756] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:45.639403 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a67a7-a731-4b89-a146-6fcd0d56c679 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.639689 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a67a7-a731-4b89-a146-6fcd0d56c679 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.639978 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a67a7-a731-4b89-a146-6fcd0d56c679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.672031 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-6c8a67a7-a731-4b89-a146-6fcd0d56c679 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:45.673314 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6c8a67a7-a731-4b89-a146-6fcd0d56c679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:45.681386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1380/2757] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:45.685805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33ab37-b91e-4dbc-99a9-9245768a0d2f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:45.696122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33ab37-b91e-4dbc-99a9-9245768a0d2f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:45.701090 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c2656-c208-470c-b47f-27f9b89c6f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:45.701783 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c2656-c208-470c-b47f-27f9b89c6f50 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.702065 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c2656-c208-470c-b47f-27f9b89c6f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.702344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c2656-c208-470c-b47f-27f9b89c6f50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.703078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c2656-c208-470c-b47f-27f9b89c6f50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:35:45.703525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c2656-c208-470c-b47f-27f9b89c6f50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:45.705173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c2656-c208-470c-b47f-27f9b89c6f50 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:45.719256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1378/2758] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:35:45.724953 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a67a7-a731-4b89-a146-6fcd0d56c679 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:45.733700 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a67a7-a731-4b89-a146-6fcd0d56c679 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:45.738979 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa34316-1231-4451-902d-0f8434551b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:45.740321 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa34316-1231-4451-902d-0f8434551b57 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.740619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa34316-1231-4451-902d-0f8434551b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.740900 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa34316-1231-4451-902d-0f8434551b57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.741609 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa34316-1231-4451-902d-0f8434551b57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:45.743586 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa34316-1231-4451-902d-0f8434551b57 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:45.754604 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:45.787202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1381/2759] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:35:45 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:35:45.791623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c2656-c208-470c-b47f-27f9b89c6f50 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:45.802376 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c2656-c208-470c-b47f-27f9b89c6f50 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:45.873730 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd5bc6-8f9b-404b-8df8-10f3cab61a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:45.874309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd5bc6-8f9b-404b-8df8-10f3cab61a46 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.874603 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd5bc6-8f9b-404b-8df8-10f3cab61a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.874873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd5bc6-8f9b-404b-8df8-10f3cab61a46 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.875778 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd5bc6-8f9b-404b-8df8-10f3cab61a46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:45.877902 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd5bc6-8f9b-404b-8df8-10f3cab61a46 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:45.904716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1379/2760] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:35:45 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:35:45.906568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa34316-1231-4451-902d-0f8434551b57 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:45.919561 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa34316-1231-4451-902d-0f8434551b57 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:45.925178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240f9cc-42da-4379-83b6-19ae47f04754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:45.925930 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240f9cc-42da-4379-83b6-19ae47f04754 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.926278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240f9cc-42da-4379-83b6-19ae47f04754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.926611 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240f9cc-42da-4379-83b6-19ae47f04754 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.927416 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240f9cc-42da-4379-83b6-19ae47f04754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:45.929246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240f9cc-42da-4379-83b6-19ae47f04754 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:45.952773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1382/2761] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:35:45.956980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd5bc6-8f9b-404b-8df8-10f3cab61a46 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:45.974487 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd5bc6-8f9b-404b-8df8-10f3cab61a46 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:45.978342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843810b-f304-4dd1-b8fc-0eba368f842d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:45.979021 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843810b-f304-4dd1-b8fc-0eba368f842d admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.979311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843810b-f304-4dd1-b8fc-0eba368f842d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.979594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843810b-f304-4dd1-b8fc-0eba368f842d admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.980363 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843810b-f304-4dd1-b8fc-0eba368f842d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:45.982507 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843810b-f304-4dd1-b8fc-0eba368f842d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:45.993450 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1380/2762] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:35:45.997722 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240f9cc-42da-4379-83b6-19ae47f04754 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:46.010241 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240f9cc-42da-4379-83b6-19ae47f04754 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:46.017127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1cbb2-0554-4dc9-84cf-e0652a13fd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:46.018706 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1cbb2-0554-4dc9-84cf-e0652a13fd45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:46.019217 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1cbb2-0554-4dc9-84cf-e0652a13fd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:46.019723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1cbb2-0554-4dc9-84cf-e0652a13fd45 admin admin] PATH_INFO: `/v3/projects/b2e72a1efa7c48aaa036ea2c739c6c21/users/3bd7300087a844b586caf081c5f0fba2/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:46.039372 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1cbb2-0554-4dc9-84cf-e0652a13fd45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2e72a1efa7c48aaa036ea2c739c6c21, user_id=3bd7300087a844b586caf081c5f0fba2, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:46.042515 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1cbb2-0554-4dc9-84cf-e0652a13fd45 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:46.061625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1383/2763] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:35:45 2022] PUT /identity/v3/projects/b2e72a1efa7c48aaa036ea2c739c6c21/users/3bd7300087a844b586caf081c5f0fba2/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:46.074135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487980c-d696-4bad-aeba-5a10a06df470 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:46.074443 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487980c-d696-4bad-aeba-5a10a06df470 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:46.074732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487980c-d696-4bad-aeba-5a10a06df470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:46.106396 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7487980c-d696-4bad-aeba-5a10a06df470 None None] MFA Rules not processed for user `3bd7300087a844b586caf081c5f0fba2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:46.107708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7487980c-d696-4bad-aeba-5a10a06df470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:46.181925 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1381/2764] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:35:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:35:46.199283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1cbb2-0554-4dc9-84cf-e0652a13fd45 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:46.202041 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-dff1cbb2-0554-4dc9-84cf-e0652a13fd45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:46.292240 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1cbb2-0554-4dc9-84cf-e0652a13fd45 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:46.299437 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170312f-5676-4773-88eb-51811f70686d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:46.300139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170312f-5676-4773-88eb-51811f70686d service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:46.300410 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170312f-5676-4773-88eb-51811f70686d service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:46.300682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170312f-5676-4773-88eb-51811f70686d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:46.309723 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170312f-5676-4773-88eb-51811f70686d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:46.311657 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170312f-5676-4773-88eb-51811f70686d service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:46.321450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1384/2765] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:35:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:35:53.588478 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487980c-d696-4bad-aeba-5a10a06df470 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:53.633489 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:53.651950 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487980c-d696-4bad-aeba-5a10a06df470 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:53.656654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35386009-58de-4ca0-b0e6-550a882d9ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:53.657578 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35386009-58de-4ca0-b0e6-550a882d9ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:53.657896 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35386009-58de-4ca0-b0e6-550a882d9ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:53.658176 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35386009-58de-4ca0-b0e6-550a882d9ef0 admin admin] PATH_INFO: `/v3/users/3bd7300087a844b586caf081c5f0fba2` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:53.659418 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35386009-58de-4ca0-b0e6-550a882d9ef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bd7300087a844b586caf081c5f0fba2)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:53.661474 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35386009-58de-4ca0-b0e6-550a882d9ef0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:53.707061 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-35386009-58de-4ca0-b0e6-550a882d9ef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bd7300087a844b586caf081c5f0fba2'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:53.714328 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-35386009-58de-4ca0-b0e6-550a882d9ef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bd7300087a844b586caf081c5f0fba2'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:53.714950 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-35386009-58de-4ca0-b0e6-550a882d9ef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bd7300087a844b586caf081c5f0fba2'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:53.721906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-35386009-58de-4ca0-b0e6-550a882d9ef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bd7300087a844b586caf081c5f0fba2'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:53.733551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-35386009-58de-4ca0-b0e6-550a882d9ef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bd7300087a844b586caf081c5f0fba2'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:53.739992 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1382/2766] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:35:53 2022] DELETE /identity/v3/users/3bd7300087a844b586caf081c5f0fba2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:53.744699 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2ffcd-f2bb-4853-952c-250d148cde88 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:53.745182 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2ffcd-f2bb-4853-952c-250d148cde88 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:53.745283 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2ffcd-f2bb-4853-952c-250d148cde88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:53.762506 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-57b2ffcd-f2bb-4853-952c-250d148cde88 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:53.763751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-57b2ffcd-f2bb-4853-952c-250d148cde88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:53.822965 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1385/2767] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:53.832650 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35386009-58de-4ca0-b0e6-550a882d9ef0 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:53.834051 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-35386009-58de-4ca0-b0e6-550a882d9ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:53.890810 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35386009-58de-4ca0-b0e6-550a882d9ef0 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:53.894692 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf38ce-fdd9-4e5f-a142-9d2d6b0423e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:53.895399 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf38ce-fdd9-4e5f-a142-9d2d6b0423e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:53.895689 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf38ce-fdd9-4e5f-a142-9d2d6b0423e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:53.895994 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf38ce-fdd9-4e5f-a142-9d2d6b0423e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:53.905674 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf38ce-fdd9-4e5f-a142-9d2d6b0423e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:53.907104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf38ce-fdd9-4e5f-a142-9d2d6b0423e4 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:53.916766 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1383/2768] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:35:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:35:54.183069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2ffcd-f2bb-4853-952c-250d148cde88 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:54.184640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-57b2ffcd-f2bb-4853-952c-250d148cde88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:54.197099 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2ffcd-f2bb-4853-952c-250d148cde88 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:54.201767 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8c607-1eca-4d68-94d4-47ef5b0c1c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:54.202641 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8c607-1eca-4d68-94d4-47ef5b0c1c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:54.202924 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8c607-1eca-4d68-94d4-47ef5b0c1c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:54.203202 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8c607-1eca-4d68-94d4-47ef5b0c1c9b admin admin] PATH_INFO: `/v3/projects/b2e72a1efa7c48aaa036ea2c739c6c21` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:54.204811 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8c607-1eca-4d68-94d4-47ef5b0c1c9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2e72a1efa7c48aaa036ea2c739c6c21)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:54.206776 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8c607-1eca-4d68-94d4-47ef5b0c1c9b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:54.240052 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:54.256555 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4cb8c607-1eca-4d68-94d4-47ef5b0c1c9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2e72a1efa7c48aaa036ea2c739c6c21', 'request_id': 'req-4cb8c607-1eca-4d68-94d4-47ef5b0c1c9b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:54.265299 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4cb8c607-1eca-4d68-94d4-47ef5b0c1c9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2e72a1efa7c48aaa036ea2c739c6c21', 'request_id': 'req-4cb8c607-1eca-4d68-94d4-47ef5b0c1c9b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:54.274771 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4cb8c607-1eca-4d68-94d4-47ef5b0c1c9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2e72a1efa7c48aaa036ea2c739c6c21', 'request_id': 'req-4cb8c607-1eca-4d68-94d4-47ef5b0c1c9b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:54.285796 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4cb8c607-1eca-4d68-94d4-47ef5b0c1c9b admin admin] The token cache is being invalidate because project b2e72a1efa7c48aaa036ea2c739c6c21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:35:54.286578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4cb8c607-1eca-4d68-94d4-47ef5b0c1c9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:54.287223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-4cb8c607-1eca-4d68-94d4-47ef5b0c1c9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:54.288802 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1386/2769] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:35:54 2022] DELETE /identity/v3/projects/b2e72a1efa7c48aaa036ea2c739c6c21 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:54.294144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c8441-72d9-476c-b893-2242c9f53ccb None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:54.294435 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c8441-72d9-476c-b893-2242c9f53ccb None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:54.294715 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c8441-72d9-476c-b893-2242c9f53ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:54.312844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1e0c8441-72d9-476c-b893-2242c9f53ccb None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:54.314036 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1e0c8441-72d9-476c-b893-2242c9f53ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:54.374263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1384/2770] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:54.382490 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2fd20-8523-4cf7-90b7-fe824aa82b42 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:54.382490 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2fd20-8523-4cf7-90b7-fe824aa82b42 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:54.382763 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2fd20-8523-4cf7-90b7-fe824aa82b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:54.400981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-86f2fd20-8523-4cf7-90b7-fe824aa82b42 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:54.402226 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-86f2fd20-8523-4cf7-90b7-fe824aa82b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:54.410084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1387/2771] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:54.414925 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c8441-72d9-476c-b893-2242c9f53ccb None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:54.424665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c8441-72d9-476c-b893-2242c9f53ccb None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:54.430154 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ca978-be75-4136-8eb4-46d6bba16c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:54.430747 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ca978-be75-4136-8eb4-46d6bba16c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:54.431030 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ca978-be75-4136-8eb4-46d6bba16c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:54.431311 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ca978-be75-4136-8eb4-46d6bba16c7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:54.432038 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ca978-be75-4136-8eb4-46d6bba16c7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:35:54.432378 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ca978-be75-4136-8eb4-46d6bba16c7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:54.434028 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ca978-be75-4136-8eb4-46d6bba16c7b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:54.435523 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:54.447826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1385/2772] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:35:54.452021 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2fd20-8523-4cf7-90b7-fe824aa82b42 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:54.461927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2fd20-8523-4cf7-90b7-fe824aa82b42 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:54.466641 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbc887-17cc-4f1e-a0a0-13d46803174f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:54.467292 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbc887-17cc-4f1e-a0a0-13d46803174f admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:54.467578 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbc887-17cc-4f1e-a0a0-13d46803174f admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:54.467883 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbc887-17cc-4f1e-a0a0-13d46803174f admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:54.468620 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbc887-17cc-4f1e-a0a0-13d46803174f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:54.470498 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbc887-17cc-4f1e-a0a0-13d46803174f admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:54.495483 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1388/2773] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:35:54 2022] POST /identity/v3/projects => generated 403 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:35:54.499566 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ca978-be75-4136-8eb4-46d6bba16c7b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:54.510058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ca978-be75-4136-8eb4-46d6bba16c7b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:54.566215 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449128fc-35f8-49ed-8153-a9e7ab6030f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:54.566880 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449128fc-35f8-49ed-8153-a9e7ab6030f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:54.567258 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449128fc-35f8-49ed-8153-a9e7ab6030f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:54.567538 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449128fc-35f8-49ed-8153-a9e7ab6030f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:54.568240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449128fc-35f8-49ed-8153-a9e7ab6030f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:54.570186 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449128fc-35f8-49ed-8153-a9e7ab6030f7 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:54.590476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1386/2774] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:35:54 2022] POST /identity/v3/users => generated 302 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:35:54.594752 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbc887-17cc-4f1e-a0a0-13d46803174f admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:54.605709 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbc887-17cc-4f1e-a0a0-13d46803174f admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:54.610451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961b9cd-f010-43fd-9858-62f8756fd5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:54.611104 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961b9cd-f010-43fd-9858-62f8756fd5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:54.611386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961b9cd-f010-43fd-9858-62f8756fd5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:54.611661 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961b9cd-f010-43fd-9858-62f8756fd5f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:54.612429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961b9cd-f010-43fd-9858-62f8756fd5f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:54.614003 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961b9cd-f010-43fd-9858-62f8756fd5f1 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:54.624209 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1389/2775] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:35:54.628156 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449128fc-35f8-49ed-8153-a9e7ab6030f7 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:54.638057 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449128fc-35f8-49ed-8153-a9e7ab6030f7 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:54.642626 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe60a3-02ac-4a9f-b61b-908871e35e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:54.643278 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe60a3-02ac-4a9f-b61b-908871e35e79 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:54.643557 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe60a3-02ac-4a9f-b61b-908871e35e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:54.643835 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe60a3-02ac-4a9f-b61b-908871e35e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:54.644671 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe60a3-02ac-4a9f-b61b-908871e35e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:54.646782 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe60a3-02ac-4a9f-b61b-908871e35e79 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:54.656867 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1387/2776] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:35:54.661638 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961b9cd-f010-43fd-9858-62f8756fd5f1 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:54.671601 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961b9cd-f010-43fd-9858-62f8756fd5f1 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:54.676255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7461182-6609-4ac1-b9fb-832295507d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:54.677167 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7461182-6609-4ac1-b9fb-832295507d47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:54.677455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7461182-6609-4ac1-b9fb-832295507d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:54.677732 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7461182-6609-4ac1-b9fb-832295507d47 admin admin] PATH_INFO: `/v3/projects/d426cba4a9e145c1aef615a32d8ad465/users/8799edff86bd44908373838ed707b030/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:54.692096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7461182-6609-4ac1-b9fb-832295507d47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d426cba4a9e145c1aef615a32d8ad465, user_id=8799edff86bd44908373838ed707b030, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:54.694735 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7461182-6609-4ac1-b9fb-832295507d47 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:54.707462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1390/2777] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:35:54 2022] PUT /identity/v3/projects/d426cba4a9e145c1aef615a32d8ad465/users/8799edff86bd44908373838ed707b030/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:54.719361 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04cb47-3210-441f-99d0-aad214115012 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:54.719650 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04cb47-3210-441f-99d0-aad214115012 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:54.719962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04cb47-3210-441f-99d0-aad214115012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:54.746437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-cb04cb47-3210-441f-99d0-aad214115012 None None] MFA Rules not processed for user `8799edff86bd44908373838ed707b030`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:54.747645 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cb04cb47-3210-441f-99d0-aad214115012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:54.813364 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1388/2778] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:35:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:54.818996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7461182-6609-4ac1-b9fb-832295507d47 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:54.885652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7461182-6609-4ac1-b9fb-832295507d47 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:54.890366 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bc775-73d0-4cf0-baef-b439d8a63361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:54.891025 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bc775-73d0-4cf0-baef-b439d8a63361 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:54.891310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bc775-73d0-4cf0-baef-b439d8a63361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:54.891583 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bc775-73d0-4cf0-baef-b439d8a63361 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:54.892276 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81bc775-73d0-4cf0-baef-b439d8a63361 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:54.894184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81bc775-73d0-4cf0-baef-b439d8a63361 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:54.919874 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1391/2779] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:35:54 2022] POST /identity/v3/projects => generated 403 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:35:54.924539 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04cb47-3210-441f-99d0-aad214115012 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:54.935157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04cb47-3210-441f-99d0-aad214115012 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:54.991980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff2461d-2d45-4adf-94db-1224c29c3991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:54.992586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2461d-2d45-4adf-94db-1224c29c3991 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:54.992885 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2461d-2d45-4adf-94db-1224c29c3991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:54.993147 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2461d-2d45-4adf-94db-1224c29c3991 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:54.993837 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2461d-2d45-4adf-94db-1224c29c3991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:54.995748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2461d-2d45-4adf-94db-1224c29c3991 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:55.003302 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:55.016589 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1389/2780] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:35:54 2022] POST /identity/v3/users => generated 308 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:35:55.021136 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bc775-73d0-4cf0-baef-b439d8a63361 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:55.031277 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bc775-73d0-4cf0-baef-b439d8a63361 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:55.035952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d0d22-5aa7-4252-9835-e4ac68252f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:55.036644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d0d22-5aa7-4252-9835-e4ac68252f60 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:55.036933 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d0d22-5aa7-4252-9835-e4ac68252f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:55.037215 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d0d22-5aa7-4252-9835-e4ac68252f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:55.038008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d0d22-5aa7-4252-9835-e4ac68252f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:55.039640 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d0d22-5aa7-4252-9835-e4ac68252f60 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:55.050489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1392/2781] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:35:55.054672 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff2461d-2d45-4adf-94db-1224c29c3991 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:55.065152 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff2461d-2d45-4adf-94db-1224c29c3991 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:55.070826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c84a5f-b9e9-49a7-82e7-049aa37af6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:55.071703 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c84a5f-b9e9-49a7-82e7-049aa37af6b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:55.071984 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c84a5f-b9e9-49a7-82e7-049aa37af6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:55.072260 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c84a5f-b9e9-49a7-82e7-049aa37af6b9 admin admin] PATH_INFO: `/v3/projects/0ccdfabea07348999bb8d08210ff2220/users/3beb79a49c334d549c57b05aa7d1fae8/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:55.087575 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c84a5f-b9e9-49a7-82e7-049aa37af6b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ccdfabea07348999bb8d08210ff2220, user_id=3beb79a49c334d549c57b05aa7d1fae8, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:55.090247 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c84a5f-b9e9-49a7-82e7-049aa37af6b9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:55.103662 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1390/2782] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:35:55 2022] PUT /identity/v3/projects/0ccdfabea07348999bb8d08210ff2220/users/3beb79a49c334d549c57b05aa7d1fae8/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:55.107362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d0d22-5aa7-4252-9835-e4ac68252f60 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:55.118204 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d0d22-5aa7-4252-9835-e4ac68252f60 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:55.144205 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:55.182896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade48713-696e-4841-9baf-9459e7bf88e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:55.183745 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade48713-696e-4841-9baf-9459e7bf88e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:55.184040 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade48713-696e-4841-9baf-9459e7bf88e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:55.184417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade48713-696e-4841-9baf-9459e7bf88e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:55.185160 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade48713-696e-4841-9baf-9459e7bf88e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:55.187008 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade48713-696e-4841-9baf-9459e7bf88e3 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:55.198234 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1393/2783] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:35:55.202268 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c84a5f-b9e9-49a7-82e7-049aa37af6b9 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:55.213007 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c84a5f-b9e9-49a7-82e7-049aa37af6b9 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:55.217796 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d09c3-b110-4760-8739-119946ec4d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:55.218688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d09c3-b110-4760-8739-119946ec4d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:55.218972 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d09c3-b110-4760-8739-119946ec4d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:55.219249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d09c3-b110-4760-8739-119946ec4d7d admin admin] PATH_INFO: `/v3/projects/0ccdfabea07348999bb8d08210ff2220/users/3beb79a49c334d549c57b05aa7d1fae8/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:55.221257 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d09c3-b110-4760-8739-119946ec4d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ccdfabea07348999bb8d08210ff2220, user_id=3beb79a49c334d549c57b05aa7d1fae8, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:55.223916 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d09c3-b110-4760-8739-119946ec4d7d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:55.234339 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1391/2784] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:35:55 2022] PUT /identity/v3/projects/0ccdfabea07348999bb8d08210ff2220/users/3beb79a49c334d549c57b05aa7d1fae8/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:35:55.244997 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b38ca-447c-403a-9f0b-0d044e48d001 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:55.245255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b38ca-447c-403a-9f0b-0d044e48d001 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:55.245539 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b38ca-447c-403a-9f0b-0d044e48d001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:55.272212 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-880b38ca-447c-403a-9f0b-0d044e48d001 None None] MFA Rules not processed for user `3beb79a49c334d549c57b05aa7d1fae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:55.273458 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-880b38ca-447c-403a-9f0b-0d044e48d001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:55.342037 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1394/2785] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:35:55 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:55.353059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d09c3-b110-4760-8739-119946ec4d7d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:55.354766 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ad4d09c3-b110-4760-8739-119946ec4d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:55.411237 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d09c3-b110-4760-8739-119946ec4d7d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:55.424131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff6da9-feb8-4e42-b355-556827cd61cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:55.424131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff6da9-feb8-4e42-b355-556827cd61cb service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:55.424131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff6da9-feb8-4e42-b355-556827cd61cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:55.424131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff6da9-feb8-4e42-b355-556827cd61cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:55.431948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff6da9-feb8-4e42-b355-556827cd61cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:55.433422 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff6da9-feb8-4e42-b355-556827cd61cb service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:55.443650 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1392/2786] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:35:55 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:35:55.578773 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b38ca-447c-403a-9f0b-0d044e48d001 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:55.763828 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b38ca-447c-403a-9f0b-0d044e48d001 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:55.768149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4e032-f478-4a2e-8f85-680b49627f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:55.769047 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4e032-f478-4a2e-8f85-680b49627f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:55.769326 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4e032-f478-4a2e-8f85-680b49627f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:55.769602 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4e032-f478-4a2e-8f85-680b49627f96 admin admin] PATH_INFO: `/v3/users/8799edff86bd44908373838ed707b030` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:55.770897 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4e032-f478-4a2e-8f85-680b49627f96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8799edff86bd44908373838ed707b030)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:55.772834 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4e032-f478-4a2e-8f85-680b49627f96 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:55.821045 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-92a4e032-f478-4a2e-8f85-680b49627f96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8799edff86bd44908373838ed707b030'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:55.823526 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:55.828804 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-92a4e032-f478-4a2e-8f85-680b49627f96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8799edff86bd44908373838ed707b030'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:55.841653 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-92a4e032-f478-4a2e-8f85-680b49627f96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8799edff86bd44908373838ed707b030'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:55.848913 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-92a4e032-f478-4a2e-8f85-680b49627f96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8799edff86bd44908373838ed707b030'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:55.849614 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-92a4e032-f478-4a2e-8f85-680b49627f96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8799edff86bd44908373838ed707b030'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:55.855570 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1395/2787] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:35:55 2022] DELETE /identity/v3/users/8799edff86bd44908373838ed707b030 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:55.860080 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d111e28-9156-476d-b301-1624c58cbd21 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:55.860678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d111e28-9156-476d-b301-1624c58cbd21 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:55.860678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d111e28-9156-476d-b301-1624c58cbd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:55.878701 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3d111e28-9156-476d-b301-1624c58cbd21 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:55.879915 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3d111e28-9156-476d-b301-1624c58cbd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:55.938598 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1393/2788] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:55.948308 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4e032-f478-4a2e-8f85-680b49627f96 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:55.949743 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-92a4e032-f478-4a2e-8f85-680b49627f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:56.006852 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4e032-f478-4a2e-8f85-680b49627f96 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:56.011284 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05de02-788e-4627-b086-7f87b4a7daa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:56.012193 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05de02-788e-4627-b086-7f87b4a7daa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:56.012621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05de02-788e-4627-b086-7f87b4a7daa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:56.012996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05de02-788e-4627-b086-7f87b4a7daa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:56.024995 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05de02-788e-4627-b086-7f87b4a7daa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:56.026359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05de02-788e-4627-b086-7f87b4a7daa7 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:56.037917 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1396/2789] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:35:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:35:56.302585 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d111e28-9156-476d-b301-1624c58cbd21 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:56.304071 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3d111e28-9156-476d-b301-1624c58cbd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:56.317952 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d111e28-9156-476d-b301-1624c58cbd21 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:56.322294 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4de30-017b-4047-87c6-3dbd2496846d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:56.323161 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4de30-017b-4047-87c6-3dbd2496846d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:56.323445 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4de30-017b-4047-87c6-3dbd2496846d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:56.323721 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4de30-017b-4047-87c6-3dbd2496846d admin admin] PATH_INFO: `/v3/projects/d426cba4a9e145c1aef615a32d8ad465` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:56.324983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4de30-017b-4047-87c6-3dbd2496846d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d426cba4a9e145c1aef615a32d8ad465)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:56.326881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4de30-017b-4047-87c6-3dbd2496846d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:56.377093 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b8c4de30-017b-4047-87c6-3dbd2496846d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd426cba4a9e145c1aef615a32d8ad465', 'request_id': 'req-b8c4de30-017b-4047-87c6-3dbd2496846d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:56.378948 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:INSERT=4,keystone:SELECT=92,keystone:DELETE=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:56.385610 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b8c4de30-017b-4047-87c6-3dbd2496846d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd426cba4a9e145c1aef615a32d8ad465', 'request_id': 'req-b8c4de30-017b-4047-87c6-3dbd2496846d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:56.392674 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b8c4de30-017b-4047-87c6-3dbd2496846d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd426cba4a9e145c1aef615a32d8ad465', 'request_id': 'req-b8c4de30-017b-4047-87c6-3dbd2496846d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:56.403832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b8c4de30-017b-4047-87c6-3dbd2496846d admin admin] The token cache is being invalidate because project d426cba4a9e145c1aef615a32d8ad465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:35:56.404113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b8c4de30-017b-4047-87c6-3dbd2496846d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:56.404769 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b8c4de30-017b-4047-87c6-3dbd2496846d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:56.406312 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1394/2790] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:35:56 2022] DELETE /identity/v3/projects/d426cba4a9e145c1aef615a32d8ad465 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:56.410553 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05de02-788e-4627-b086-7f87b4a7daa7 service neutron] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:56.420127 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05de02-788e-4627-b086-7f87b4a7daa7 service neutron] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:56.475232 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29697f-1ce9-442f-a4d5-b8e6e3d6f632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:56.476098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29697f-1ce9-442f-a4d5-b8e6e3d6f632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:56.476408 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29697f-1ce9-442f-a4d5-b8e6e3d6f632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:56.476681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29697f-1ce9-442f-a4d5-b8e6e3d6f632 admin admin] PATH_INFO: `/v3/users/3beb79a49c334d549c57b05aa7d1fae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:56.477949 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29697f-1ce9-442f-a4d5-b8e6e3d6f632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3beb79a49c334d549c57b05aa7d1fae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:56.479825 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29697f-1ce9-442f-a4d5-b8e6e3d6f632 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:56.524420 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ee29697f-1ce9-442f-a4d5-b8e6e3d6f632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3beb79a49c334d549c57b05aa7d1fae8'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:56.531777 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ee29697f-1ce9-442f-a4d5-b8e6e3d6f632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3beb79a49c334d549c57b05aa7d1fae8'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:56.544128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ee29697f-1ce9-442f-a4d5-b8e6e3d6f632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3beb79a49c334d549c57b05aa7d1fae8'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:56.545440 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:56.552635 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ee29697f-1ce9-442f-a4d5-b8e6e3d6f632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3beb79a49c334d549c57b05aa7d1fae8'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:56.553603 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-ee29697f-1ce9-442f-a4d5-b8e6e3d6f632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3beb79a49c334d549c57b05aa7d1fae8'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:56.560481 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1397/2791] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:35:56 2022] DELETE /identity/v3/users/3beb79a49c334d549c57b05aa7d1fae8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:56.835982 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4de30-017b-4047-87c6-3dbd2496846d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:56.837505 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b8c4de30-017b-4047-87c6-3dbd2496846d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:56.850689 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4de30-017b-4047-87c6-3dbd2496846d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:56.910769 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24e8ce-2d29-4cf7-b3cd-75bfc699862f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:56.911646 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24e8ce-2d29-4cf7-b3cd-75bfc699862f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:56.911928 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24e8ce-2d29-4cf7-b3cd-75bfc699862f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:56.912203 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24e8ce-2d29-4cf7-b3cd-75bfc699862f admin admin] PATH_INFO: `/v3/projects/0ccdfabea07348999bb8d08210ff2220` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:56.913438 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24e8ce-2d29-4cf7-b3cd-75bfc699862f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ccdfabea07348999bb8d08210ff2220)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:56.915761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24e8ce-2d29-4cf7-b3cd-75bfc699862f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:56.964780 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-db24e8ce-2d29-4cf7-b3cd-75bfc699862f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ccdfabea07348999bb8d08210ff2220', 'request_id': 'req-db24e8ce-2d29-4cf7-b3cd-75bfc699862f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:56.970840 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-db24e8ce-2d29-4cf7-b3cd-75bfc699862f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ccdfabea07348999bb8d08210ff2220', 'request_id': 'req-db24e8ce-2d29-4cf7-b3cd-75bfc699862f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:56.978965 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-db24e8ce-2d29-4cf7-b3cd-75bfc699862f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ccdfabea07348999bb8d08210ff2220', 'request_id': 'req-db24e8ce-2d29-4cf7-b3cd-75bfc699862f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:56.990407 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-db24e8ce-2d29-4cf7-b3cd-75bfc699862f admin admin] The token cache is being invalidate because project 0ccdfabea07348999bb8d08210ff2220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:35:56.990708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-db24e8ce-2d29-4cf7-b3cd-75bfc699862f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:56.991368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-db24e8ce-2d29-4cf7-b3cd-75bfc699862f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:56.993608 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1395/2792] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:35:56 2022] DELETE /identity/v3/projects/0ccdfabea07348999bb8d08210ff2220 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:56.999186 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c25a15-8fee-49dc-86d2-e7a0c7a61965 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:56.999719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c25a15-8fee-49dc-86d2-e7a0c7a61965 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:56.999719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c25a15-8fee-49dc-86d2-e7a0c7a61965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.018457 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-53c25a15-8fee-49dc-86d2-e7a0c7a61965 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:57.019698 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-53c25a15-8fee-49dc-86d2-e7a0c7a61965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:57.079817 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1398/2793] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:57.088784 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd977c-3dc3-4b11-8c59-e539ceb7d592 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.089078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd977c-3dc3-4b11-8c59-e539ceb7d592 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.089349 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd977c-3dc3-4b11-8c59-e539ceb7d592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.107785 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0ecd977c-3dc3-4b11-8c59-e539ceb7d592 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:57.108998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0ecd977c-3dc3-4b11-8c59-e539ceb7d592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:57.118820 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1396/2794] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:57.122882 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c25a15-8fee-49dc-86d2-e7a0c7a61965 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:57.132818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c25a15-8fee-49dc-86d2-e7a0c7a61965 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:57.138495 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36763717-7c76-445d-bff1-04804677a1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:57.139150 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36763717-7c76-445d-bff1-04804677a1e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.139429 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36763717-7c76-445d-bff1-04804677a1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.139703 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36763717-7c76-445d-bff1-04804677a1e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.140470 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36763717-7c76-445d-bff1-04804677a1e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:35:57.140780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36763717-7c76-445d-bff1-04804677a1e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:57.142348 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36763717-7c76-445d-bff1-04804677a1e4 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:57.154691 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1399/2795] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:35:57.159179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd977c-3dc3-4b11-8c59-e539ceb7d592 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:57.168962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd977c-3dc3-4b11-8c59-e539ceb7d592 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:57.173774 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0753a6-b9d6-4e14-be52-9a622a34a1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:57.174417 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0753a6-b9d6-4e14-be52-9a622a34a1bf admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.174694 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0753a6-b9d6-4e14-be52-9a622a34a1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.174977 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0753a6-b9d6-4e14-be52-9a622a34a1bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.175663 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0753a6-b9d6-4e14-be52-9a622a34a1bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:57.178196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0753a6-b9d6-4e14-be52-9a622a34a1bf admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:57.203130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1397/2796] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:35:57 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:35:57.207273 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36763717-7c76-445d-bff1-04804677a1e4 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:57.218128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36763717-7c76-445d-bff1-04804677a1e4 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:57.275007 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ce346-1277-44ee-b5f3-894614c25942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:57.275705 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ce346-1277-44ee-b5f3-894614c25942 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.275990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ce346-1277-44ee-b5f3-894614c25942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.276265 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ce346-1277-44ee-b5f3-894614c25942 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.276991 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ce346-1277-44ee-b5f3-894614c25942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:57.278888 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ce346-1277-44ee-b5f3-894614c25942 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:57.299585 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1400/2797] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:35:57 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:35:57.304576 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0753a6-b9d6-4e14-be52-9a622a34a1bf admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:57.315326 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0753a6-b9d6-4e14-be52-9a622a34a1bf admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:57.319956 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d216fe-12b3-4251-ac8d-3a88dcb08a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:57.320640 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d216fe-12b3-4251-ac8d-3a88dcb08a15 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.320929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d216fe-12b3-4251-ac8d-3a88dcb08a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.321206 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d216fe-12b3-4251-ac8d-3a88dcb08a15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.321929 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d216fe-12b3-4251-ac8d-3a88dcb08a15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:57.323554 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d216fe-12b3-4251-ac8d-3a88dcb08a15 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:57.334184 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1398/2798] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:35:57.338296 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ce346-1277-44ee-b5f3-894614c25942 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:57.348950 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ce346-1277-44ee-b5f3-894614c25942 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:57.353682 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a42f1-4450-4b2f-8ff5-7692eca283c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:57.354386 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a42f1-4450-4b2f-8ff5-7692eca283c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.354623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a42f1-4450-4b2f-8ff5-7692eca283c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.354898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a42f1-4450-4b2f-8ff5-7692eca283c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.355628 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a42f1-4450-4b2f-8ff5-7692eca283c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:57.357240 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a42f1-4450-4b2f-8ff5-7692eca283c6 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:57.368152 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1401/2799] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:35:57.372621 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d216fe-12b3-4251-ac8d-3a88dcb08a15 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:57.382805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d216fe-12b3-4251-ac8d-3a88dcb08a15 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:57.387711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67548953-7650-48fa-86c1-916a260aba67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:57.388650 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67548953-7650-48fa-86c1-916a260aba67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.388943 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67548953-7650-48fa-86c1-916a260aba67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.389224 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67548953-7650-48fa-86c1-916a260aba67 admin admin] PATH_INFO: `/v3/projects/b8f3b18ce04542e78eb4e7ab63f3f64b/users/5df65abe8b934395862b75ad7c0b3fb4/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.406291 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67548953-7650-48fa-86c1-916a260aba67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8f3b18ce04542e78eb4e7ab63f3f64b, user_id=5df65abe8b934395862b75ad7c0b3fb4, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:57.409049 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67548953-7650-48fa-86c1-916a260aba67 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:57.425764 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1399/2800] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:35:57 2022] PUT /identity/v3/projects/b8f3b18ce04542e78eb4e7ab63f3f64b/users/5df65abe8b934395862b75ad7c0b3fb4/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:35:57.436880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31beafd3-8429-47a8-91c3-c561aca12c3b None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.437173 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31beafd3-8429-47a8-91c3-c561aca12c3b None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.437455 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31beafd3-8429-47a8-91c3-c561aca12c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.463753 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-31beafd3-8429-47a8-91c3-c561aca12c3b None None] MFA Rules not processed for user `5df65abe8b934395862b75ad7c0b3fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:57.464997 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-31beafd3-8429-47a8-91c3-c561aca12c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:57.509292 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:57.531463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1402/2801] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:35:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:57.536551 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67548953-7650-48fa-86c1-916a260aba67 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:57.545570 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:57.603615 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67548953-7650-48fa-86c1-916a260aba67 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:57.608216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926465d3-7987-42a3-a9ef-b820e67f4654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:57.608906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926465d3-7987-42a3-a9ef-b820e67f4654 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.609190 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926465d3-7987-42a3-a9ef-b820e67f4654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.609494 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926465d3-7987-42a3-a9ef-b820e67f4654 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.610192 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926465d3-7987-42a3-a9ef-b820e67f4654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:57.612111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926465d3-7987-42a3-a9ef-b820e67f4654 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:57.640532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1400/2802] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:35:57 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:35:57.644562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31beafd3-8429-47a8-91c3-c561aca12c3b None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:57.656057 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31beafd3-8429-47a8-91c3-c561aca12c3b None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:57.712656 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545cac8-0132-4258-bd03-c05a073ae390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:57.713351 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545cac8-0132-4258-bd03-c05a073ae390 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.713633 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545cac8-0132-4258-bd03-c05a073ae390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.713910 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545cac8-0132-4258-bd03-c05a073ae390 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.714596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545cac8-0132-4258-bd03-c05a073ae390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:57.716560 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545cac8-0132-4258-bd03-c05a073ae390 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:57.736442 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1403/2803] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:35:57 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:35:57.740586 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926465d3-7987-42a3-a9ef-b820e67f4654 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:57.750775 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926465d3-7987-42a3-a9ef-b820e67f4654 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:57.755415 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376d3b4-e677-4cc3-ab7b-6e3d1391e3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:57.756078 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376d3b4-e677-4cc3-ab7b-6e3d1391e3e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.756403 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376d3b4-e677-4cc3-ab7b-6e3d1391e3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.756684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376d3b4-e677-4cc3-ab7b-6e3d1391e3e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.757411 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376d3b4-e677-4cc3-ab7b-6e3d1391e3e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:57.758993 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376d3b4-e677-4cc3-ab7b-6e3d1391e3e1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:57.769398 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1401/2804] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:35:57.773911 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545cac8-0132-4258-bd03-c05a073ae390 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:57.783734 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545cac8-0132-4258-bd03-c05a073ae390 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:57.788877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235803f6-4bb1-4dca-bde6-6354f8bd9364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:57.789757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235803f6-4bb1-4dca-bde6-6354f8bd9364 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.790048 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235803f6-4bb1-4dca-bde6-6354f8bd9364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.790327 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235803f6-4bb1-4dca-bde6-6354f8bd9364 admin admin] PATH_INFO: `/v3/projects/01bf54d1b4f24173983f1e35fae1702f/users/7d4cca258c46480084d79ee5c4243ac3/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.805087 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235803f6-4bb1-4dca-bde6-6354f8bd9364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01bf54d1b4f24173983f1e35fae1702f, user_id=7d4cca258c46480084d79ee5c4243ac3, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:57.807707 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235803f6-4bb1-4dca-bde6-6354f8bd9364 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:57.821902 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1404/2805] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:35:57 2022] PUT /identity/v3/projects/01bf54d1b4f24173983f1e35fae1702f/users/7d4cca258c46480084d79ee5c4243ac3/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:57.825880 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376d3b4-e677-4cc3-ab7b-6e3d1391e3e1 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:57.836591 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376d3b4-e677-4cc3-ab7b-6e3d1391e3e1 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:57.903627 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592ad6d-34bc-4a64-a10a-ebd9c894adf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:57.904196 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592ad6d-34bc-4a64-a10a-ebd9c894adf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.904512 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592ad6d-34bc-4a64-a10a-ebd9c894adf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.904792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592ad6d-34bc-4a64-a10a-ebd9c894adf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.905532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592ad6d-34bc-4a64-a10a-ebd9c894adf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:57.907129 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592ad6d-34bc-4a64-a10a-ebd9c894adf8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:57.917017 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1402/2806] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:35:57.921130 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235803f6-4bb1-4dca-bde6-6354f8bd9364 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:57.933642 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235803f6-4bb1-4dca-bde6-6354f8bd9364 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:57.938474 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54602b5d-ecef-4a2e-8151-6cf4fb9d2da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:57.939344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54602b5d-ecef-4a2e-8151-6cf4fb9d2da0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.939623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54602b5d-ecef-4a2e-8151-6cf4fb9d2da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.939927 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54602b5d-ecef-4a2e-8151-6cf4fb9d2da0 admin admin] PATH_INFO: `/v3/projects/01bf54d1b4f24173983f1e35fae1702f/users/7d4cca258c46480084d79ee5c4243ac3/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.941922 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54602b5d-ecef-4a2e-8151-6cf4fb9d2da0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01bf54d1b4f24173983f1e35fae1702f, user_id=7d4cca258c46480084d79ee5c4243ac3, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:57.945042 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54602b5d-ecef-4a2e-8151-6cf4fb9d2da0 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:57.955086 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1405/2807] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:35:57 2022] PUT /identity/v3/projects/01bf54d1b4f24173983f1e35fae1702f/users/7d4cca258c46480084d79ee5c4243ac3/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:57.965898 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598bdf-d274-446b-852a-74940303a94b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.966192 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598bdf-d274-446b-852a-74940303a94b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.966467 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598bdf-d274-446b-852a-74940303a94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.992113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2f598bdf-d274-446b-852a-74940303a94b None None] MFA Rules not processed for user `7d4cca258c46480084d79ee5c4243ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:57.993437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2f598bdf-d274-446b-852a-74940303a94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:58.065725 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1403/2808] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:35:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:58.077572 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54602b5d-ecef-4a2e-8151-6cf4fb9d2da0 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:58.079296 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-54602b5d-ecef-4a2e-8151-6cf4fb9d2da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:58.138491 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54602b5d-ecef-4a2e-8151-6cf4fb9d2da0 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:58.142139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1857a-a3cd-4a33-8b16-1281562da3ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:58.142778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1857a-a3cd-4a33-8b16-1281562da3ae service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:58.143087 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1857a-a3cd-4a33-8b16-1281562da3ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:58.143365 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1857a-a3cd-4a33-8b16-1281562da3ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:58.152751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1857a-a3cd-4a33-8b16-1281562da3ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:58.154220 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1857a-a3cd-4a33-8b16-1281562da3ae service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:58.163778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1406/2809] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:35:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:35:58.562635 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f598bdf-d274-446b-852a-74940303a94b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:58.566866 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:58.630176 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f598bdf-d274-446b-852a-74940303a94b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:58.634925 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3770fe6-7c5b-48db-b436-437e83b8d4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:58.636018 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3770fe6-7c5b-48db-b436-437e83b8d4f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:58.636018 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3770fe6-7c5b-48db-b436-437e83b8d4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:58.636018 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3770fe6-7c5b-48db-b436-437e83b8d4f0 admin admin] PATH_INFO: `/v3/users/5df65abe8b934395862b75ad7c0b3fb4` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:58.638182 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3770fe6-7c5b-48db-b436-437e83b8d4f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5df65abe8b934395862b75ad7c0b3fb4)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:58.639587 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3770fe6-7c5b-48db-b436-437e83b8d4f0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:58.691832 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a3770fe6-7c5b-48db-b436-437e83b8d4f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5df65abe8b934395862b75ad7c0b3fb4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.699231 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a3770fe6-7c5b-48db-b436-437e83b8d4f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5df65abe8b934395862b75ad7c0b3fb4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.700013 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a3770fe6-7c5b-48db-b436-437e83b8d4f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5df65abe8b934395862b75ad7c0b3fb4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.707220 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a3770fe6-7c5b-48db-b436-437e83b8d4f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5df65abe8b934395862b75ad7c0b3fb4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.718999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a3770fe6-7c5b-48db-b436-437e83b8d4f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5df65abe8b934395862b75ad7c0b3fb4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.725315 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1404/2810] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:35:58 2022] DELETE /identity/v3/users/5df65abe8b934395862b75ad7c0b3fb4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:58.729922 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbddd7-faec-4d42-9030-fa948e280ab2 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:58.730220 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbddd7-faec-4d42-9030-fa948e280ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:58.730525 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbddd7-faec-4d42-9030-fa948e280ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:58.750432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-43dbddd7-faec-4d42-9030-fa948e280ab2 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:58.751644 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-43dbddd7-faec-4d42-9030-fa948e280ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:58.783312 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:58.809490 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1407/2811] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:58.819407 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3770fe6-7c5b-48db-b436-437e83b8d4f0 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:58.821580 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a3770fe6-7c5b-48db-b436-437e83b8d4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:58.882729 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3770fe6-7c5b-48db-b436-437e83b8d4f0 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:58.886408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b367e7f-9338-4c46-824f-db29ca4133fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:58.887047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b367e7f-9338-4c46-824f-db29ca4133fc service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:58.887327 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b367e7f-9338-4c46-824f-db29ca4133fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:58.887599 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b367e7f-9338-4c46-824f-db29ca4133fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:58.896954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b367e7f-9338-4c46-824f-db29ca4133fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:58.898494 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b367e7f-9338-4c46-824f-db29ca4133fc service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:58.908478 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1405/2812] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:35:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:35:59.183446 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbddd7-faec-4d42-9030-fa948e280ab2 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:59.185358 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-43dbddd7-faec-4d42-9030-fa948e280ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:59.198196 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbddd7-faec-4d42-9030-fa948e280ab2 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:59.202909 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb3b4c-e845-425b-9d28-c322c0af8f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:59.204014 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb3b4c-e845-425b-9d28-c322c0af8f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:59.204314 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb3b4c-e845-425b-9d28-c322c0af8f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:59.204600 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb3b4c-e845-425b-9d28-c322c0af8f37 admin admin] PATH_INFO: `/v3/projects/b8f3b18ce04542e78eb4e7ab63f3f64b` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.205822 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb3b4c-e845-425b-9d28-c322c0af8f37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8f3b18ce04542e78eb4e7ab63f3f64b)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:59.208768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb3b4c-e845-425b-9d28-c322c0af8f37 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:59.256188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5cbb3b4c-e845-425b-9d28-c322c0af8f37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8f3b18ce04542e78eb4e7ab63f3f64b', 'request_id': 'req-5cbb3b4c-e845-425b-9d28-c322c0af8f37'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:59.264362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5cbb3b4c-e845-425b-9d28-c322c0af8f37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8f3b18ce04542e78eb4e7ab63f3f64b', 'request_id': 'req-5cbb3b4c-e845-425b-9d28-c322c0af8f37'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:59.269899 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5cbb3b4c-e845-425b-9d28-c322c0af8f37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8f3b18ce04542e78eb4e7ab63f3f64b', 'request_id': 'req-5cbb3b4c-e845-425b-9d28-c322c0af8f37'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:59.281718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5cbb3b4c-e845-425b-9d28-c322c0af8f37 admin admin] The token cache is being invalidate because project b8f3b18ce04542e78eb4e7ab63f3f64b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:35:59.282009 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5cbb3b4c-e845-425b-9d28-c322c0af8f37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:59.282687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-5cbb3b4c-e845-425b-9d28-c322c0af8f37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:59.284219 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1408/2813] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:35:59 2022] DELETE /identity/v3/projects/b8f3b18ce04542e78eb4e7ab63f3f64b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:59.288829 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b367e7f-9338-4c46-824f-db29ca4133fc service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:59.298596 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b367e7f-9338-4c46-824f-db29ca4133fc service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:59.319084 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:59.363148 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428ea5f-2afa-4301-bfe4-7036c754f4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:59.364143 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428ea5f-2afa-4301-bfe4-7036c754f4a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:59.364461 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428ea5f-2afa-4301-bfe4-7036c754f4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:59.364740 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428ea5f-2afa-4301-bfe4-7036c754f4a5 admin admin] PATH_INFO: `/v3/users/7d4cca258c46480084d79ee5c4243ac3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.366042 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428ea5f-2afa-4301-bfe4-7036c754f4a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d4cca258c46480084d79ee5c4243ac3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:59.367918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428ea5f-2afa-4301-bfe4-7036c754f4a5 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:59.420401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3428ea5f-2afa-4301-bfe4-7036c754f4a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d4cca258c46480084d79ee5c4243ac3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:59.428043 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3428ea5f-2afa-4301-bfe4-7036c754f4a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d4cca258c46480084d79ee5c4243ac3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:59.428801 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3428ea5f-2afa-4301-bfe4-7036c754f4a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d4cca258c46480084d79ee5c4243ac3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:59.436804 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3428ea5f-2afa-4301-bfe4-7036c754f4a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d4cca258c46480084d79ee5c4243ac3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:59.451038 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3428ea5f-2afa-4301-bfe4-7036c754f4a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d4cca258c46480084d79ee5c4243ac3'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:59.457170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1406/2814] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:35:59 2022] DELETE /identity/v3/users/7d4cca258c46480084d79ee5c4243ac3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:59.737826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb3b4c-e845-425b-9d28-c322c0af8f37 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:59.739768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-5cbb3b4c-e845-425b-9d28-c322c0af8f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:59.753032 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb3b4c-e845-425b-9d28-c322c0af8f37 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:59.811810 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd7719-0c58-43e0-8dee-b61a6607b945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:59.812619 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd7719-0c58-43e0-8dee-b61a6607b945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:59.812901 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd7719-0c58-43e0-8dee-b61a6607b945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:59.813173 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd7719-0c58-43e0-8dee-b61a6607b945 admin admin] PATH_INFO: `/v3/projects/01bf54d1b4f24173983f1e35fae1702f` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.814346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd7719-0c58-43e0-8dee-b61a6607b945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01bf54d1b4f24173983f1e35fae1702f)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:59.816209 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd7719-0c58-43e0-8dee-b61a6607b945 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:59.861885 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-53fd7719-0c58-43e0-8dee-b61a6607b945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01bf54d1b4f24173983f1e35fae1702f', 'request_id': 'req-53fd7719-0c58-43e0-8dee-b61a6607b945'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:59.869833 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-53fd7719-0c58-43e0-8dee-b61a6607b945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01bf54d1b4f24173983f1e35fae1702f', 'request_id': 'req-53fd7719-0c58-43e0-8dee-b61a6607b945'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:59.875438 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-53fd7719-0c58-43e0-8dee-b61a6607b945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01bf54d1b4f24173983f1e35fae1702f', 'request_id': 'req-53fd7719-0c58-43e0-8dee-b61a6607b945'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:59.876532 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:59.889330 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-53fd7719-0c58-43e0-8dee-b61a6607b945 admin admin] The token cache is being invalidate because project 01bf54d1b4f24173983f1e35fae1702f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:35:59.889623 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-53fd7719-0c58-43e0-8dee-b61a6607b945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:59.890265 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-53fd7719-0c58-43e0-8dee-b61a6607b945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:59.891838 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1409/2815] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:35:59 2022] DELETE /identity/v3/projects/01bf54d1b4f24173983f1e35fae1702f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:59.896976 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9480d5-23a1-41d9-bbd8-5e17e5e0636a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:59.897272 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9480d5-23a1-41d9-bbd8-5e17e5e0636a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:59.897557 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9480d5-23a1-41d9-bbd8-5e17e5e0636a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.914769 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ca9480d5-23a1-41d9-bbd8-5e17e5e0636a None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:59.915986 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ca9480d5-23a1-41d9-bbd8-5e17e5e0636a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:59.975480 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1407/2816] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:00.035065 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48363640-1c07-4643-b88b-0208bb4465f5 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.035354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48363640-1c07-4643-b88b-0208bb4465f5 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.035637 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48363640-1c07-4643-b88b-0208bb4465f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.054253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-48363640-1c07-4643-b88b-0208bb4465f5 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:00.055454 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-48363640-1c07-4643-b88b-0208bb4465f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:00.063604 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1410/2817] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:00.068386 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9480d5-23a1-41d9-bbd8-5e17e5e0636a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:00.078113 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9480d5-23a1-41d9-bbd8-5e17e5e0636a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:00.082745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083e7e1-c92d-43de-8055-ed12ac4979c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:00.083404 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083e7e1-c92d-43de-8055-ed12ac4979c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.083691 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083e7e1-c92d-43de-8055-ed12ac4979c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.083969 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083e7e1-c92d-43de-8055-ed12ac4979c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.084745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083e7e1-c92d-43de-8055-ed12ac4979c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:36:00.085059 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083e7e1-c92d-43de-8055-ed12ac4979c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:00.086647 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083e7e1-c92d-43de-8055-ed12ac4979c6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:00.098652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1408/2818] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:36:00.102944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48363640-1c07-4643-b88b-0208bb4465f5 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:00.112063 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48363640-1c07-4643-b88b-0208bb4465f5 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:00.117347 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c14bea-797b-4b4d-9980-51092217bb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:00.117988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c14bea-797b-4b4d-9980-51092217bb6c admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.118270 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c14bea-797b-4b4d-9980-51092217bb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.118547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c14bea-797b-4b4d-9980-51092217bb6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.119243 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c14bea-797b-4b4d-9980-51092217bb6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:00.121178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c14bea-797b-4b4d-9980-51092217bb6c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:00.146837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1411/2819] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:00 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:00.151344 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083e7e1-c92d-43de-8055-ed12ac4979c6 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:00.162015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083e7e1-c92d-43de-8055-ed12ac4979c6 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:00.191236 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:00.221346 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fea6ea-79d1-443a-9384-93dbb8a7a4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:00.222011 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fea6ea-79d1-443a-9384-93dbb8a7a4aa admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.222289 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fea6ea-79d1-443a-9384-93dbb8a7a4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.222563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fea6ea-79d1-443a-9384-93dbb8a7a4aa admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.223744 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fea6ea-79d1-443a-9384-93dbb8a7a4aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:00.225768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fea6ea-79d1-443a-9384-93dbb8a7a4aa admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:00.245930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1409/2820] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:00 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:00.250144 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c14bea-797b-4b4d-9980-51092217bb6c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:00.260430 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c14bea-797b-4b4d-9980-51092217bb6c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:00.265032 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831dbb1-0255-4cd7-ae81-a6ce332a464b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:00.265687 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831dbb1-0255-4cd7-ae81-a6ce332a464b admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.265970 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831dbb1-0255-4cd7-ae81-a6ce332a464b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.266246 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831dbb1-0255-4cd7-ae81-a6ce332a464b admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.266972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831dbb1-0255-4cd7-ae81-a6ce332a464b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:00.268575 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831dbb1-0255-4cd7-ae81-a6ce332a464b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:00.278826 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1412/2821] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:00.282778 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fea6ea-79d1-443a-9384-93dbb8a7a4aa admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:00.293327 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fea6ea-79d1-443a-9384-93dbb8a7a4aa admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:00.298013 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50726c-0a95-4799-b979-d282633382ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:00.298669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50726c-0a95-4799-b979-d282633382ed admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.298957 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50726c-0a95-4799-b979-d282633382ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.299238 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50726c-0a95-4799-b979-d282633382ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.299980 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50726c-0a95-4799-b979-d282633382ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:00.302221 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50726c-0a95-4799-b979-d282633382ed admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:00.312240 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1410/2822] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:00.316333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831dbb1-0255-4cd7-ae81-a6ce332a464b admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:00.326391 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831dbb1-0255-4cd7-ae81-a6ce332a464b admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:00.330995 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b5d55-b999-4e47-998e-9d2b45e91b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:00.331872 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b5d55-b999-4e47-998e-9d2b45e91b98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.332153 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b5d55-b999-4e47-998e-9d2b45e91b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.332482 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b5d55-b999-4e47-998e-9d2b45e91b98 admin admin] PATH_INFO: `/v3/projects/4a81425d5d31454281e7eb36ac9f5302/users/a5c95bb024b34356aed6a029b0f6b61c/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.349372 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b5d55-b999-4e47-998e-9d2b45e91b98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a81425d5d31454281e7eb36ac9f5302, user_id=a5c95bb024b34356aed6a029b0f6b61c, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:00.352093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b5d55-b999-4e47-998e-9d2b45e91b98 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:00.365741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1413/2823] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:00 2022] PUT /identity/v3/projects/4a81425d5d31454281e7eb36ac9f5302/users/a5c95bb024b34356aed6a029b0f6b61c/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:00.375777 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144014d-65ab-48ae-9fc0-5ac926d02990 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.376072 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144014d-65ab-48ae-9fc0-5ac926d02990 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.376399 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144014d-65ab-48ae-9fc0-5ac926d02990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.402144 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1144014d-65ab-48ae-9fc0-5ac926d02990 None None] MFA Rules not processed for user `a5c95bb024b34356aed6a029b0f6b61c`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:00.403358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1144014d-65ab-48ae-9fc0-5ac926d02990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:00.468482 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1411/2824] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:36:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:00.472903 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b5d55-b999-4e47-998e-9d2b45e91b98 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:00.533255 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b5d55-b999-4e47-998e-9d2b45e91b98 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:00.542984 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bfd85-1520-4a97-abcd-c789ba9371b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:00.544064 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bfd85-1520-4a97-abcd-c789ba9371b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.544596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bfd85-1520-4a97-abcd-c789ba9371b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.545067 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bfd85-1520-4a97-abcd-c789ba9371b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.546353 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969bfd85-1520-4a97-abcd-c789ba9371b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:00.549930 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969bfd85-1520-4a97-abcd-c789ba9371b7 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:00.589880 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1414/2825] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:00 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:00.593930 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144014d-65ab-48ae-9fc0-5ac926d02990 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:00.605241 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144014d-65ab-48ae-9fc0-5ac926d02990 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:00.661904 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d68ccd-8656-438a-bc7b-5b34fe7279ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:00.662741 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d68ccd-8656-438a-bc7b-5b34fe7279ca admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.663032 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d68ccd-8656-438a-bc7b-5b34fe7279ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.663309 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d68ccd-8656-438a-bc7b-5b34fe7279ca admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.663999 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d68ccd-8656-438a-bc7b-5b34fe7279ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:00.665946 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d68ccd-8656-438a-bc7b-5b34fe7279ca admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:00.687684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1412/2826] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:00 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:00.691792 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bfd85-1520-4a97-abcd-c789ba9371b7 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:00.702068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bfd85-1520-4a97-abcd-c789ba9371b7 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:00.706652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ec8f7-21b9-4b9e-9675-6bd5474cad28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:00.707306 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ec8f7-21b9-4b9e-9675-6bd5474cad28 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.708098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ec8f7-21b9-4b9e-9675-6bd5474cad28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.708417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ec8f7-21b9-4b9e-9675-6bd5474cad28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.709146 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ec8f7-21b9-4b9e-9675-6bd5474cad28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:00.710724 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ec8f7-21b9-4b9e-9675-6bd5474cad28 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:00.721652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1415/2827] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:00.726205 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d68ccd-8656-438a-bc7b-5b34fe7279ca admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:00.736037 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d68ccd-8656-438a-bc7b-5b34fe7279ca admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:00.740643 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820bc22e-6c39-4ff8-a137-3004c35c0631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:00.741290 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bc22e-6c39-4ff8-a137-3004c35c0631 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.741571 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bc22e-6c39-4ff8-a137-3004c35c0631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.741846 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bc22e-6c39-4ff8-a137-3004c35c0631 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.742577 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820bc22e-6c39-4ff8-a137-3004c35c0631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:00.744150 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820bc22e-6c39-4ff8-a137-3004c35c0631 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:00.754011 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1413/2828] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:00.758006 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ec8f7-21b9-4b9e-9675-6bd5474cad28 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:00.767654 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ec8f7-21b9-4b9e-9675-6bd5474cad28 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:00.772759 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c5050-640f-4fb7-b234-06aee4858d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:00.773559 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c5050-640f-4fb7-b234-06aee4858d58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.773837 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c5050-640f-4fb7-b234-06aee4858d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.774108 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c5050-640f-4fb7-b234-06aee4858d58 admin admin] PATH_INFO: `/v3/projects/4b1c1b05af6f468c9487e7aeddf3ce90/users/f4d3fcd49488456dae2baca523f27f06/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.788712 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c5050-640f-4fb7-b234-06aee4858d58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b1c1b05af6f468c9487e7aeddf3ce90, user_id=f4d3fcd49488456dae2baca523f27f06, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:00.791305 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c5050-640f-4fb7-b234-06aee4858d58 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:00.804003 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1416/2829] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:00 2022] PUT /identity/v3/projects/4b1c1b05af6f468c9487e7aeddf3ce90/users/f4d3fcd49488456dae2baca523f27f06/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:36:00.814841 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d222537-b81e-4ec7-aad1-cbead7822364 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.815208 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d222537-b81e-4ec7-aad1-cbead7822364 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.815558 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d222537-b81e-4ec7-aad1-cbead7822364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.844165 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4d222537-b81e-4ec7-aad1-cbead7822364 None None] MFA Rules not processed for user `f4d3fcd49488456dae2baca523f27f06`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:00.845331 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4d222537-b81e-4ec7-aad1-cbead7822364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:00.850541 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:00.912391 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1414/2830] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:36:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:00.917012 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c5050-640f-4fb7-b234-06aee4858d58 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:00.968900 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:00.989521 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c5050-640f-4fb7-b234-06aee4858d58 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:00.994198 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7a002-7449-466c-86dd-264ba0e65952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:00.994856 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7a002-7449-466c-86dd-264ba0e65952 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.995142 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7a002-7449-466c-86dd-264ba0e65952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.995421 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7a002-7449-466c-86dd-264ba0e65952 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.996113 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7a002-7449-466c-86dd-264ba0e65952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:00.998027 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7a002-7449-466c-86dd-264ba0e65952 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:01.023832 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1417/2831] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:00 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:01.027958 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d222537-b81e-4ec7-aad1-cbead7822364 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:01.039026 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d222537-b81e-4ec7-aad1-cbead7822364 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:01.094671 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a6e59-b3bc-466f-b9db-308a67c2e569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:01.095260 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a6e59-b3bc-466f-b9db-308a67c2e569 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:01.095547 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a6e59-b3bc-466f-b9db-308a67c2e569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:01.095828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a6e59-b3bc-466f-b9db-308a67c2e569 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:01.096559 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a6e59-b3bc-466f-b9db-308a67c2e569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:01.098541 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a6e59-b3bc-466f-b9db-308a67c2e569 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:01.118424 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1415/2832] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:01 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:01.122208 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7a002-7449-466c-86dd-264ba0e65952 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:01.132961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7a002-7449-466c-86dd-264ba0e65952 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:01.137565 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9282d2-104e-4689-bba7-0e055e84474e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:01.138213 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9282d2-104e-4689-bba7-0e055e84474e admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:01.138499 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9282d2-104e-4689-bba7-0e055e84474e admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:01.138778 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9282d2-104e-4689-bba7-0e055e84474e admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:01.139612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9282d2-104e-4689-bba7-0e055e84474e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:01.141254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9282d2-104e-4689-bba7-0e055e84474e admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:01.153248 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1418/2833] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:01.158449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a6e59-b3bc-466f-b9db-308a67c2e569 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:01.168861 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a6e59-b3bc-466f-b9db-308a67c2e569 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:01.174688 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46456f20-a795-4c9a-a18b-cf60e8eb861a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:01.175628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46456f20-a795-4c9a-a18b-cf60e8eb861a admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:01.175911 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46456f20-a795-4c9a-a18b-cf60e8eb861a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:01.176743 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46456f20-a795-4c9a-a18b-cf60e8eb861a admin admin] PATH_INFO: `/v3/projects/96fe7d4131314a2e8808cbf2122cc8e2/users/53bd38c8664943aa987864c271ceb0a9/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:01.193857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46456f20-a795-4c9a-a18b-cf60e8eb861a admin admin] RBAC: Authorizing `identity:create_grant(project_id=96fe7d4131314a2e8808cbf2122cc8e2, user_id=53bd38c8664943aa987864c271ceb0a9, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:01.196560 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46456f20-a795-4c9a-a18b-cf60e8eb861a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:01.209287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1416/2834] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:01 2022] PUT /identity/v3/projects/96fe7d4131314a2e8808cbf2122cc8e2/users/53bd38c8664943aa987864c271ceb0a9/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:01.213465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9282d2-104e-4689-bba7-0e055e84474e admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:01.224001 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9282d2-104e-4689-bba7-0e055e84474e admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:01.279055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85c057-3f76-4ab8-95af-e1c5a0af68e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:01.279715 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85c057-3f76-4ab8-95af-e1c5a0af68e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:01.279990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85c057-3f76-4ab8-95af-e1c5a0af68e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:01.280272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85c057-3f76-4ab8-95af-e1c5a0af68e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:01.281026 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85c057-3f76-4ab8-95af-e1c5a0af68e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:01.282598 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85c057-3f76-4ab8-95af-e1c5a0af68e1 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:01.291955 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1419/2835] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:01.296407 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46456f20-a795-4c9a-a18b-cf60e8eb861a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:01.307262 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46456f20-a795-4c9a-a18b-cf60e8eb861a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:01.312076 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1cc6f-452f-4d27-a773-c7d4c629783f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:01.312975 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1cc6f-452f-4d27-a773-c7d4c629783f admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:01.313256 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1cc6f-452f-4d27-a773-c7d4c629783f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:01.313526 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1cc6f-452f-4d27-a773-c7d4c629783f admin admin] PATH_INFO: `/v3/projects/96fe7d4131314a2e8808cbf2122cc8e2/users/53bd38c8664943aa987864c271ceb0a9/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:01.315517 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1cc6f-452f-4d27-a773-c7d4c629783f admin admin] RBAC: Authorizing `identity:create_grant(project_id=96fe7d4131314a2e8808cbf2122cc8e2, user_id=53bd38c8664943aa987864c271ceb0a9, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:01.318813 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1cc6f-452f-4d27-a773-c7d4c629783f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:01.328700 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1417/2836] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:01 2022] PUT /identity/v3/projects/96fe7d4131314a2e8808cbf2122cc8e2/users/53bd38c8664943aa987864c271ceb0a9/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:01.340122 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abba806-acd7-4cab-b031-439cd6451139 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:01.340450 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abba806-acd7-4cab-b031-439cd6451139 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:01.340734 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abba806-acd7-4cab-b031-439cd6451139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:01.367344 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-0abba806-acd7-4cab-b031-439cd6451139 None None] MFA Rules not processed for user `53bd38c8664943aa987864c271ceb0a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:01.368607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0abba806-acd7-4cab-b031-439cd6451139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:01.434486 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1420/2837] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:36:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:01.446711 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1cc6f-452f-4d27-a773-c7d4c629783f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:01.448440 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-86e1cc6f-452f-4d27-a773-c7d4c629783f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:01.510358 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1cc6f-452f-4d27-a773-c7d4c629783f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:01.514036 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fba77-f41b-4b6b-bb07-7bb207955487 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:01.514669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fba77-f41b-4b6b-bb07-7bb207955487 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:01.514955 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fba77-f41b-4b6b-bb07-7bb207955487 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:01.515229 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fba77-f41b-4b6b-bb07-7bb207955487 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:01.529765 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:01.578716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fba77-f41b-4b6b-bb07-7bb207955487 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:01.580151 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fba77-f41b-4b6b-bb07-7bb207955487 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:01.589242 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1418/2838] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:03.080378 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abba806-acd7-4cab-b031-439cd6451139 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:03.081796 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0abba806-acd7-4cab-b031-439cd6451139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:03.093756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abba806-acd7-4cab-b031-439cd6451139 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:03.097263 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a644f6-4ef6-4ae7-a835-2ebbe2e1437e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:03.097902 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a644f6-4ef6-4ae7-a835-2ebbe2e1437e service cinder] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:03.098713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a644f6-4ef6-4ae7-a835-2ebbe2e1437e service cinder] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:03.098994 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a644f6-4ef6-4ae7-a835-2ebbe2e1437e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:03.132643 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:03.164097 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a644f6-4ef6-4ae7-a835-2ebbe2e1437e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:03.165481 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a644f6-4ef6-4ae7-a835-2ebbe2e1437e service cinder] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:03.174492 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1421/2839] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:03.719708 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fba77-f41b-4b6b-bb07-7bb207955487 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:03.729545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fba77-f41b-4b6b-bb07-7bb207955487 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:03.733218 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626faae-9f32-41b1-966d-54afa478f989 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69aef294bb4943c3822ad7d0a3d91fd5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:03.733860 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626faae-9f32-41b1-966d-54afa478f989 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:03.734149 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626faae-9f32-41b1-966d-54afa478f989 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:03.734428 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626faae-9f32-41b1-966d-54afa478f989 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:03.744170 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626faae-9f32-41b1-966d-54afa478f989 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:03.745696 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626faae-9f32-41b1-966d-54afa478f989 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:03.755250 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1419/2840] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:06.765468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a644f6-4ef6-4ae7-a835-2ebbe2e1437e service cinder] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:06.766875 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-d8a644f6-4ef6-4ae7-a835-2ebbe2e1437e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:06.871436 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a644f6-4ef6-4ae7-a835-2ebbe2e1437e service cinder] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:06.878787 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e554fa3-1b15-4521-a2ce-bbf3c84bab91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:06.880332 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e554fa3-1b15-4521-a2ce-bbf3c84bab91 service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:06.880489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e554fa3-1b15-4521-a2ce-bbf3c84bab91 service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:06.880713 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e554fa3-1b15-4521-a2ce-bbf3c84bab91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:06.883137 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6e554fa3-1b15-4521-a2ce-bbf3c84bab91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:06.910639 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e554fa3-1b15-4521-a2ce-bbf3c84bab91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:06.912559 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e554fa3-1b15-4521-a2ce-bbf3c84bab91 service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:06.932555 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1422/2841] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:07.819340 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626faae-9f32-41b1-966d-54afa478f989 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:07.883376 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626faae-9f32-41b1-966d-54afa478f989 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:07.888008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f992f2-5206-4ae5-ac4d-de4bd22f80a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:07.888912 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f992f2-5206-4ae5-ac4d-de4bd22f80a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:07.889199 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f992f2-5206-4ae5-ac4d-de4bd22f80a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:07.889478 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f992f2-5206-4ae5-ac4d-de4bd22f80a1 admin admin] PATH_INFO: `/v3/users/a5c95bb024b34356aed6a029b0f6b61c` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:07.890676 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f992f2-5206-4ae5-ac4d-de4bd22f80a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5c95bb024b34356aed6a029b0f6b61c)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:07.892819 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f992f2-5206-4ae5-ac4d-de4bd22f80a1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:07.939694 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-33f992f2-5206-4ae5-ac4d-de4bd22f80a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5c95bb024b34356aed6a029b0f6b61c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:07.946468 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-33f992f2-5206-4ae5-ac4d-de4bd22f80a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5c95bb024b34356aed6a029b0f6b61c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:07.947166 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-33f992f2-5206-4ae5-ac4d-de4bd22f80a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5c95bb024b34356aed6a029b0f6b61c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:07.954385 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-33f992f2-5206-4ae5-ac4d-de4bd22f80a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5c95bb024b34356aed6a029b0f6b61c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:07.966657 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-33f992f2-5206-4ae5-ac4d-de4bd22f80a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5c95bb024b34356aed6a029b0f6b61c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:07.972634 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1420/2842] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:07 2022] DELETE /identity/v3/users/a5c95bb024b34356aed6a029b0f6b61c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:07.977749 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831f37a-e6e6-4e5f-a343-1df7b4bd6251 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:07.978243 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831f37a-e6e6-4e5f-a343-1df7b4bd6251 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:07.978355 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831f37a-e6e6-4e5f-a343-1df7b4bd6251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:07.995335 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-c831f37a-e6e6-4e5f-a343-1df7b4bd6251 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:07.996592 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c831f37a-e6e6-4e5f-a343-1df7b4bd6251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:08.056738 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1423/2843] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:08.066617 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f992f2-5206-4ae5-ac4d-de4bd22f80a1 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:08.068036 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-33f992f2-5206-4ae5-ac4d-de4bd22f80a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:08.094561 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:08.139690 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f992f2-5206-4ae5-ac4d-de4bd22f80a1 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:08.143428 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1d6e1-5839-4f31-80de-4f0d24c7ddb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:08.144088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1d6e1-5839-4f31-80de-4f0d24c7ddb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:08.144402 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1d6e1-5839-4f31-80de-4f0d24c7ddb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:08.145191 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1d6e1-5839-4f31-80de-4f0d24c7ddb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:08.155350 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1d6e1-5839-4f31-80de-4f0d24c7ddb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:08.156913 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1d6e1-5839-4f31-80de-4f0d24c7ddb7 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:08.172047 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1421/2844] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:08.464892 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831f37a-e6e6-4e5f-a343-1df7b4bd6251 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:08.466398 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c831f37a-e6e6-4e5f-a343-1df7b4bd6251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:08.478668 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831f37a-e6e6-4e5f-a343-1df7b4bd6251 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:08.483468 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd435f-e094-4e33-a1bf-945a34c1b320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:08.484400 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd435f-e094-4e33-a1bf-945a34c1b320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:08.484694 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd435f-e094-4e33-a1bf-945a34c1b320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:08.484974 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd435f-e094-4e33-a1bf-945a34c1b320 admin admin] PATH_INFO: `/v3/projects/4a81425d5d31454281e7eb36ac9f5302` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:08.486187 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd435f-e094-4e33-a1bf-945a34c1b320 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a81425d5d31454281e7eb36ac9f5302)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:08.488094 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd435f-e094-4e33-a1bf-945a34c1b320 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:08.535930 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-98fd435f-e094-4e33-a1bf-945a34c1b320 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a81425d5d31454281e7eb36ac9f5302', 'request_id': 'req-98fd435f-e094-4e33-a1bf-945a34c1b320'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:08.539602 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:08.543568 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-98fd435f-e094-4e33-a1bf-945a34c1b320 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a81425d5d31454281e7eb36ac9f5302', 'request_id': 'req-98fd435f-e094-4e33-a1bf-945a34c1b320'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:08.550616 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-98fd435f-e094-4e33-a1bf-945a34c1b320 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a81425d5d31454281e7eb36ac9f5302', 'request_id': 'req-98fd435f-e094-4e33-a1bf-945a34c1b320'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:08.562424 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-98fd435f-e094-4e33-a1bf-945a34c1b320 admin admin] The token cache is being invalidate because project 4a81425d5d31454281e7eb36ac9f5302 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:08.562718 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-98fd435f-e094-4e33-a1bf-945a34c1b320 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:08.563389 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-98fd435f-e094-4e33-a1bf-945a34c1b320 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:08.565093 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1424/2845] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:08 2022] DELETE /identity/v3/projects/4a81425d5d31454281e7eb36ac9f5302 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:08.569667 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1d6e1-5839-4f31-80de-4f0d24c7ddb7 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:08.579867 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1d6e1-5839-4f31-80de-4f0d24c7ddb7 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:08.641193 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41737983-a6f7-4057-83b2-2bd6795f3967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:08.642070 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41737983-a6f7-4057-83b2-2bd6795f3967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:08.642350 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41737983-a6f7-4057-83b2-2bd6795f3967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:08.642638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41737983-a6f7-4057-83b2-2bd6795f3967 admin admin] PATH_INFO: `/v3/users/f4d3fcd49488456dae2baca523f27f06` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:08.643882 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41737983-a6f7-4057-83b2-2bd6795f3967 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4d3fcd49488456dae2baca523f27f06)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:08.646731 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41737983-a6f7-4057-83b2-2bd6795f3967 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:08.701433 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-41737983-a6f7-4057-83b2-2bd6795f3967 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4d3fcd49488456dae2baca523f27f06'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:08.710605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-41737983-a6f7-4057-83b2-2bd6795f3967 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4d3fcd49488456dae2baca523f27f06'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:08.711382 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-41737983-a6f7-4057-83b2-2bd6795f3967 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4d3fcd49488456dae2baca523f27f06'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:08.722264 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-41737983-a6f7-4057-83b2-2bd6795f3967 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4d3fcd49488456dae2baca523f27f06'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:08.741862 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-41737983-a6f7-4057-83b2-2bd6795f3967 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4d3fcd49488456dae2baca523f27f06'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:08.749495 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1422/2846] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:08 2022] DELETE /identity/v3/users/f4d3fcd49488456dae2baca523f27f06 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:36:09.161372 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd435f-e094-4e33-a1bf-945a34c1b320 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:09.163678 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-98fd435f-e094-4e33-a1bf-945a34c1b320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:09.184622 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd435f-e094-4e33-a1bf-945a34c1b320 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:09.265758 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e2ff3-7185-4f04-8620-a761323de38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:09.266746 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e2ff3-7185-4f04-8620-a761323de38c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:09.267035 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e2ff3-7185-4f04-8620-a761323de38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:09.267311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e2ff3-7185-4f04-8620-a761323de38c admin admin] PATH_INFO: `/v3/projects/4b1c1b05af6f468c9487e7aeddf3ce90` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:09.268737 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e2ff3-7185-4f04-8620-a761323de38c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b1c1b05af6f468c9487e7aeddf3ce90)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:09.270799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e2ff3-7185-4f04-8620-a761323de38c admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:09.336322 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6a1e2ff3-7185-4f04-8620-a761323de38c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b1c1b05af6f468c9487e7aeddf3ce90', 'request_id': 'req-6a1e2ff3-7185-4f04-8620-a761323de38c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:09.343944 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6a1e2ff3-7185-4f04-8620-a761323de38c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b1c1b05af6f468c9487e7aeddf3ce90', 'request_id': 'req-6a1e2ff3-7185-4f04-8620-a761323de38c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:09.350021 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6a1e2ff3-7185-4f04-8620-a761323de38c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b1c1b05af6f468c9487e7aeddf3ce90', 'request_id': 'req-6a1e2ff3-7185-4f04-8620-a761323de38c'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:09.360842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6a1e2ff3-7185-4f04-8620-a761323de38c admin admin] The token cache is being invalidate because project 4b1c1b05af6f468c9487e7aeddf3ce90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:09.361129 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6a1e2ff3-7185-4f04-8620-a761323de38c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:09.361768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-6a1e2ff3-7185-4f04-8620-a761323de38c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:09.363280 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1425/2847] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:09 2022] DELETE /identity/v3/projects/4b1c1b05af6f468c9487e7aeddf3ce90 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:09.367738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41737983-a6f7-4057-83b2-2bd6795f3967 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:09.369232 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-41737983-a6f7-4057-83b2-2bd6795f3967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:09.382751 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41737983-a6f7-4057-83b2-2bd6795f3967 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:09.430881 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:09.456991 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6462d8-4e88-44bd-b748-8e19665e3a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:09.458248 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6462d8-4e88-44bd-b748-8e19665e3a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:09.458603 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6462d8-4e88-44bd-b748-8e19665e3a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:09.458888 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6462d8-4e88-44bd-b748-8e19665e3a0e admin admin] PATH_INFO: `/v3/users/53bd38c8664943aa987864c271ceb0a9` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:09.467167 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6462d8-4e88-44bd-b748-8e19665e3a0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=53bd38c8664943aa987864c271ceb0a9)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:09.469225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6462d8-4e88-44bd-b748-8e19665e3a0e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:09.554476 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-df6462d8-4e88-44bd-b748-8e19665e3a0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53bd38c8664943aa987864c271ceb0a9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:09.565839 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-df6462d8-4e88-44bd-b748-8e19665e3a0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53bd38c8664943aa987864c271ceb0a9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:09.566898 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-df6462d8-4e88-44bd-b748-8e19665e3a0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53bd38c8664943aa987864c271ceb0a9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:09.579420 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-df6462d8-4e88-44bd-b748-8e19665e3a0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53bd38c8664943aa987864c271ceb0a9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:09.596030 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-df6462d8-4e88-44bd-b748-8e19665e3a0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53bd38c8664943aa987864c271ceb0a9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:09.603545 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1423/2848] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:09 2022] DELETE /identity/v3/users/53bd38c8664943aa987864c271ceb0a9 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:09.898111 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e2ff3-7185-4f04-8620-a761323de38c admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:09.904314 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-6a1e2ff3-7185-4f04-8620-a761323de38c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:09.915748 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e2ff3-7185-4f04-8620-a761323de38c admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:09.999383 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7bec6-6c07-4f5b-b5fa-081bcdf28aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:10.000252 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7bec6-6c07-4f5b-b5fa-081bcdf28aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:10.000564 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7bec6-6c07-4f5b-b5fa-081bcdf28aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:10.000839 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7bec6-6c07-4f5b-b5fa-081bcdf28aab admin admin] PATH_INFO: `/v3/projects/96fe7d4131314a2e8808cbf2122cc8e2` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:10.002068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7bec6-6c07-4f5b-b5fa-081bcdf28aab admin admin] RBAC: Authorizing `identity:delete_project(project_id=96fe7d4131314a2e8808cbf2122cc8e2)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:10.004463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7bec6-6c07-4f5b-b5fa-081bcdf28aab admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:10.021756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:10.061185 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-70c7bec6-6c07-4f5b-b5fa-081bcdf28aab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96fe7d4131314a2e8808cbf2122cc8e2', 'request_id': 'req-70c7bec6-6c07-4f5b-b5fa-081bcdf28aab'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:10.070370 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-70c7bec6-6c07-4f5b-b5fa-081bcdf28aab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96fe7d4131314a2e8808cbf2122cc8e2', 'request_id': 'req-70c7bec6-6c07-4f5b-b5fa-081bcdf28aab'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:10.079728 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-70c7bec6-6c07-4f5b-b5fa-081bcdf28aab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96fe7d4131314a2e8808cbf2122cc8e2', 'request_id': 'req-70c7bec6-6c07-4f5b-b5fa-081bcdf28aab'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:10.093873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-70c7bec6-6c07-4f5b-b5fa-081bcdf28aab admin admin] The token cache is being invalidate because project 96fe7d4131314a2e8808cbf2122cc8e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:10.094169 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-70c7bec6-6c07-4f5b-b5fa-081bcdf28aab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:10.094986 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-70c7bec6-6c07-4f5b-b5fa-081bcdf28aab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:10.096627 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1426/2849] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:09 2022] DELETE /identity/v3/projects/96fe7d4131314a2e8808cbf2122cc8e2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:10.102361 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25974f4-840a-4637-97eb-ecafa95f4595 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:10.103204 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25974f4-840a-4637-97eb-ecafa95f4595 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:10.103488 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25974f4-840a-4637-97eb-ecafa95f4595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:10.123330 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b25974f4-840a-4637-97eb-ecafa95f4595 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:10.124607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b25974f4-840a-4637-97eb-ecafa95f4595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:10.189950 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1424/2850] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:10.199204 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d2af8-f792-419e-b6b0-6b70d7e81951 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:10.199757 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d2af8-f792-419e-b6b0-6b70d7e81951 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:10.199982 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d2af8-f792-419e-b6b0-6b70d7e81951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:10.220508 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-877d2af8-f792-419e-b6b0-6b70d7e81951 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:10.221771 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-877d2af8-f792-419e-b6b0-6b70d7e81951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:10.231157 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1427/2851] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:10.236000 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25974f4-840a-4637-97eb-ecafa95f4595 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:10.246541 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25974f4-840a-4637-97eb-ecafa95f4595 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:10.251310 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f016e9e-9830-41af-bcdd-b417f0760e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:10.251965 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f016e9e-9830-41af-bcdd-b417f0760e47 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:10.252249 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f016e9e-9830-41af-bcdd-b417f0760e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:10.252556 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f016e9e-9830-41af-bcdd-b417f0760e47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:10.253286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f016e9e-9830-41af-bcdd-b417f0760e47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:36:10.253594 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f016e9e-9830-41af-bcdd-b417f0760e47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:10.255844 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f016e9e-9830-41af-bcdd-b417f0760e47 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:10.267704 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1425/2852] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:36:10.272576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d2af8-f792-419e-b6b0-6b70d7e81951 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:10.282024 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d2af8-f792-419e-b6b0-6b70d7e81951 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:10.287203 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb73fa-b042-4659-b1b7-78bcc4a97830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:10.287865 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb73fa-b042-4659-b1b7-78bcc4a97830 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:10.288149 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb73fa-b042-4659-b1b7-78bcc4a97830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:10.288465 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb73fa-b042-4659-b1b7-78bcc4a97830 admin admin] PATH_INFO: `/v3/projects` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:10.289207 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb73fa-b042-4659-b1b7-78bcc4a97830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:10.291428 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb73fa-b042-4659-b1b7-78bcc4a97830 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:10.318967 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1428/2853] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:10 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:36:10.323623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f016e9e-9830-41af-bcdd-b417f0760e47 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:10.334040 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f016e9e-9830-41af-bcdd-b417f0760e47 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:10.386638 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:10.402107 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89742f-9e99-4361-90bb-4051d3f6215a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:10.403116 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89742f-9e99-4361-90bb-4051d3f6215a admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:10.403499 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89742f-9e99-4361-90bb-4051d3f6215a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:10.403814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89742f-9e99-4361-90bb-4051d3f6215a admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:10.404913 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb89742f-9e99-4361-90bb-4051d3f6215a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:10.407706 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb89742f-9e99-4361-90bb-4051d3f6215a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:10.428452 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1426/2854] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:10 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:10.432610 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb73fa-b042-4659-b1b7-78bcc4a97830 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:10.444206 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb73fa-b042-4659-b1b7-78bcc4a97830 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:10.449075 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf8ed4-3359-4608-8757-da4fd5211d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:10.449740 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf8ed4-3359-4608-8757-da4fd5211d69 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:10.450020 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf8ed4-3359-4608-8757-da4fd5211d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:10.450292 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf8ed4-3359-4608-8757-da4fd5211d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:10.451020 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf8ed4-3359-4608-8757-da4fd5211d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:10.452788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf8ed4-3359-4608-8757-da4fd5211d69 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:10.463990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1429/2855] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:10.468201 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89742f-9e99-4361-90bb-4051d3f6215a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:10.478450 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89742f-9e99-4361-90bb-4051d3f6215a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:10.483150 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea320e8-84ce-4cfc-8b23-9a56d784d0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:10.483826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea320e8-84ce-4cfc-8b23-9a56d784d0ab admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:10.484111 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea320e8-84ce-4cfc-8b23-9a56d784d0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:10.484430 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea320e8-84ce-4cfc-8b23-9a56d784d0ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:10.485168 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea320e8-84ce-4cfc-8b23-9a56d784d0ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:10.486768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea320e8-84ce-4cfc-8b23-9a56d784d0ab admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:10.499175 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1427/2856] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:10.503625 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf8ed4-3359-4608-8757-da4fd5211d69 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:10.514552 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf8ed4-3359-4608-8757-da4fd5211d69 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:10.519196 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a5abd-2292-48d0-a75e-02356f5c6a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:10.520088 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a5abd-2292-48d0-a75e-02356f5c6a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:10.520400 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a5abd-2292-48d0-a75e-02356f5c6a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:10.520675 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a5abd-2292-48d0-a75e-02356f5c6a66 admin admin] PATH_INFO: `/v3/projects/9458b930c75e457398a48b73f88847d6/users/33bfb478a3a54fcdbef4767e36cc0b8b/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:10.536213 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a5abd-2292-48d0-a75e-02356f5c6a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9458b930c75e457398a48b73f88847d6, user_id=33bfb478a3a54fcdbef4767e36cc0b8b, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:10.538887 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a5abd-2292-48d0-a75e-02356f5c6a66 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:10.555975 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1430/2857] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:10 2022] PUT /identity/v3/projects/9458b930c75e457398a48b73f88847d6/users/33bfb478a3a54fcdbef4767e36cc0b8b/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:10.574481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2bf25-11bb-4fb1-a532-d00d694214d3 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:10.574481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2bf25-11bb-4fb1-a532-d00d694214d3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:10.574481 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2bf25-11bb-4fb1-a532-d00d694214d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:10.606235 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-66f2bf25-11bb-4fb1-a532-d00d694214d3 None None] MFA Rules not processed for user `33bfb478a3a54fcdbef4767e36cc0b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:10.607455 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-66f2bf25-11bb-4fb1-a532-d00d694214d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:10.674658 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1428/2858] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:36:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:10.687483 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a5abd-2292-48d0-a75e-02356f5c6a66 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:10.689302 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-bd3a5abd-2292-48d0-a75e-02356f5c6a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:10.788178 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a5abd-2292-48d0-a75e-02356f5c6a66 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:10.792357 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acad6b-67dd-48d0-ad3f-c1cba4e83f07 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3e28557896e41c893f0de2590bc96d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:10.793098 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acad6b-67dd-48d0-ad3f-c1cba4e83f07 service glance] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:10.793483 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acad6b-67dd-48d0-ad3f-c1cba4e83f07 service glance] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:10.793829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acad6b-67dd-48d0-ad3f-c1cba4e83f07 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:10.803758 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03acad6b-67dd-48d0-ad3f-c1cba4e83f07 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:10.805489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03acad6b-67dd-48d0-ad3f-c1cba4e83f07 service glance] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:10.815184 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1431/2859] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:13.323263 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2bf25-11bb-4fb1-a532-d00d694214d3 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:13.324752 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-66f2bf25-11bb-4fb1-a532-d00d694214d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:13.401528 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2bf25-11bb-4fb1-a532-d00d694214d3 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:13.405669 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5dcf1-6707-4cbd-8dcc-e021a5406368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:13.406547 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5dcf1-6707-4cbd-8dcc-e021a5406368 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:13.406823 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5dcf1-6707-4cbd-8dcc-e021a5406368 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:13.406954 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5dcf1-6707-4cbd-8dcc-e021a5406368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:13.409056 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-40a5dcf1-6707-4cbd-8dcc-e021a5406368 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:13.494652 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5dcf1-6707-4cbd-8dcc-e021a5406368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:13.496097 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5dcf1-6707-4cbd-8dcc-e021a5406368 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:13.518145 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1429/2860] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:17.444898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acad6b-67dd-48d0-ad3f-c1cba4e83f07 service glance] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:17.446418 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-03acad6b-67dd-48d0-ad3f-c1cba4e83f07 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:17.466171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acad6b-67dd-48d0-ad3f-c1cba4e83f07 service glance] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:17.470798 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da2c86-3c50-49ff-a51e-973e4f60368d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:17.471681 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da2c86-3c50-49ff-a51e-973e4f60368d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:17.471983 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da2c86-3c50-49ff-a51e-973e4f60368d admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:17.472253 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da2c86-3c50-49ff-a51e-973e4f60368d admin admin] PATH_INFO: `/v3/users/e1e97b329ce34e8ca7ca7d3e046a3136` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:17.474058 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da2c86-3c50-49ff-a51e-973e4f60368d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1e97b329ce34e8ca7ca7d3e046a3136)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:17.475972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da2c86-3c50-49ff-a51e-973e4f60368d admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:17.503070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:17.533348 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-47da2c86-3c50-49ff-a51e-973e4f60368d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1e97b329ce34e8ca7ca7d3e046a3136'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:17.541354 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-47da2c86-3c50-49ff-a51e-973e4f60368d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1e97b329ce34e8ca7ca7d3e046a3136'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:17.554458 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-47da2c86-3c50-49ff-a51e-973e4f60368d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1e97b329ce34e8ca7ca7d3e046a3136'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:17.561382 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-47da2c86-3c50-49ff-a51e-973e4f60368d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1e97b329ce34e8ca7ca7d3e046a3136'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:17.562078 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-47da2c86-3c50-49ff-a51e-973e4f60368d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1e97b329ce34e8ca7ca7d3e046a3136'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:17.568411 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1432/2861] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:17 2022] DELETE /identity/v3/users/e1e97b329ce34e8ca7ca7d3e046a3136 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:17.763672 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5dcf1-6707-4cbd-8dcc-e021a5406368 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:17.779898 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5dcf1-6707-4cbd-8dcc-e021a5406368 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:17.783732 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:17.845182 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb44e9-ef54-439d-8398-12db0f7b9507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:17.846064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb44e9-ef54-439d-8398-12db0f7b9507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:17.846348 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb44e9-ef54-439d-8398-12db0f7b9507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:17.846623 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb44e9-ef54-439d-8398-12db0f7b9507 admin admin] PATH_INFO: `/v3/projects/4e5d98e3bb2a4708a934a15674033c12` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:17.848540 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb44e9-ef54-439d-8398-12db0f7b9507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e5d98e3bb2a4708a934a15674033c12)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:17.850644 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb44e9-ef54-439d-8398-12db0f7b9507 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:17.900738 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f3eb44e9-ef54-439d-8398-12db0f7b9507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e5d98e3bb2a4708a934a15674033c12', 'request_id': 'req-f3eb44e9-ef54-439d-8398-12db0f7b9507'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:17.907509 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f3eb44e9-ef54-439d-8398-12db0f7b9507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e5d98e3bb2a4708a934a15674033c12', 'request_id': 'req-f3eb44e9-ef54-439d-8398-12db0f7b9507'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:17.917286 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f3eb44e9-ef54-439d-8398-12db0f7b9507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e5d98e3bb2a4708a934a15674033c12', 'request_id': 'req-f3eb44e9-ef54-439d-8398-12db0f7b9507'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:17.928807 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f3eb44e9-ef54-439d-8398-12db0f7b9507 admin admin] The token cache is being invalidate because project 4e5d98e3bb2a4708a934a15674033c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:17.929091 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f3eb44e9-ef54-439d-8398-12db0f7b9507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:17.929918 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f3eb44e9-ef54-439d-8398-12db0f7b9507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:17.931745 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1430/2862] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:17 2022] DELETE /identity/v3/projects/4e5d98e3bb2a4708a934a15674033c12 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:17.936272 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da2c86-3c50-49ff-a51e-973e4f60368d admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:17.937906 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-47da2c86-3c50-49ff-a51e-973e4f60368d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:17.950906 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da2c86-3c50-49ff-a51e-973e4f60368d admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:18.009171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac3f59-3979-4637-8560-efc30f56039d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:18.010038 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac3f59-3979-4637-8560-efc30f56039d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:18.010318 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac3f59-3979-4637-8560-efc30f56039d admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:18.010586 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac3f59-3979-4637-8560-efc30f56039d admin admin] PATH_INFO: `/v3/users/4be3bb2a783443988e8f1a07f9daecf9` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:18.011923 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac3f59-3979-4637-8560-efc30f56039d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4be3bb2a783443988e8f1a07f9daecf9)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:18.013886 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac3f59-3979-4637-8560-efc30f56039d admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:18.070489 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-efac3f59-3979-4637-8560-efc30f56039d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4be3bb2a783443988e8f1a07f9daecf9'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:18.078055 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-efac3f59-3979-4637-8560-efc30f56039d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4be3bb2a783443988e8f1a07f9daecf9'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:18.091223 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-efac3f59-3979-4637-8560-efc30f56039d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4be3bb2a783443988e8f1a07f9daecf9'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:18.098590 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-efac3f59-3979-4637-8560-efc30f56039d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4be3bb2a783443988e8f1a07f9daecf9'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:18.099285 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-efac3f59-3979-4637-8560-efc30f56039d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4be3bb2a783443988e8f1a07f9daecf9'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:18.106670 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1433/2863] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:17 2022] DELETE /identity/v3/users/4be3bb2a783443988e8f1a07f9daecf9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:18.300400 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb44e9-ef54-439d-8398-12db0f7b9507 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:18.301851 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f3eb44e9-ef54-439d-8398-12db0f7b9507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:18.315035 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb44e9-ef54-439d-8398-12db0f7b9507 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:18.382665 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259725a-3ee1-43e3-825b-939e3cf0e6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:18.383562 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259725a-3ee1-43e3-825b-939e3cf0e6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:18.383876 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259725a-3ee1-43e3-825b-939e3cf0e6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:18.384100 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259725a-3ee1-43e3-825b-939e3cf0e6b9 admin admin] PATH_INFO: `/v3/projects/35acb2515a4d449a85f918e18041d387` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:18.385326 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a259725a-3ee1-43e3-825b-939e3cf0e6b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35acb2515a4d449a85f918e18041d387)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:18.387314 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a259725a-3ee1-43e3-825b-939e3cf0e6b9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:18.429897 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:18.442729 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a259725a-3ee1-43e3-825b-939e3cf0e6b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35acb2515a4d449a85f918e18041d387', 'request_id': 'req-a259725a-3ee1-43e3-825b-939e3cf0e6b9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:18.450430 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a259725a-3ee1-43e3-825b-939e3cf0e6b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35acb2515a4d449a85f918e18041d387', 'request_id': 'req-a259725a-3ee1-43e3-825b-939e3cf0e6b9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:18.459567 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a259725a-3ee1-43e3-825b-939e3cf0e6b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35acb2515a4d449a85f918e18041d387', 'request_id': 'req-a259725a-3ee1-43e3-825b-939e3cf0e6b9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:18.470581 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a259725a-3ee1-43e3-825b-939e3cf0e6b9 admin admin] The token cache is being invalidate because project 35acb2515a4d449a85f918e18041d387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:18.470873 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a259725a-3ee1-43e3-825b-939e3cf0e6b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:18.471534 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a259725a-3ee1-43e3-825b-939e3cf0e6b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:18.473164 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1431/2864] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:18 2022] DELETE /identity/v3/projects/35acb2515a4d449a85f918e18041d387 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:28.102494 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:DELETE=19,keystone:SELECT=73,keystone:INSERT=2 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:28.467470 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=10,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:36.490315 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac3f59-3979-4637-8560-efc30f56039d admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:36.492265 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-efac3f59-3979-4637-8560-efc30f56039d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:36.504799 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac3f59-3979-4637-8560-efc30f56039d admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:36.558765 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1634966-fc60-41fe-9548-4d212bebf765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:36.559627 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1634966-fc60-41fe-9548-4d212bebf765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:36.560161 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1634966-fc60-41fe-9548-4d212bebf765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:36.560463 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1634966-fc60-41fe-9548-4d212bebf765 admin admin] PATH_INFO: `/v3/users/33bfb478a3a54fcdbef4767e36cc0b8b` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:36.561632 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1634966-fc60-41fe-9548-4d212bebf765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33bfb478a3a54fcdbef4767e36cc0b8b)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:36.563503 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1634966-fc60-41fe-9548-4d212bebf765 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:36.611889 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e1634966-fc60-41fe-9548-4d212bebf765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33bfb478a3a54fcdbef4767e36cc0b8b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:36.619542 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e1634966-fc60-41fe-9548-4d212bebf765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33bfb478a3a54fcdbef4767e36cc0b8b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:36.631950 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e1634966-fc60-41fe-9548-4d212bebf765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33bfb478a3a54fcdbef4767e36cc0b8b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:36.638939 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e1634966-fc60-41fe-9548-4d212bebf765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33bfb478a3a54fcdbef4767e36cc0b8b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:36.639607 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-e1634966-fc60-41fe-9548-4d212bebf765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33bfb478a3a54fcdbef4767e36cc0b8b'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:36.645588 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1434/2865] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:36 2022] DELETE /identity/v3/users/33bfb478a3a54fcdbef4767e36cc0b8b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:36.650073 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab8e9d-7c70-40e9-ab0d-aa40404ec91e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:36.650570 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab8e9d-7c70-40e9-ab0d-aa40404ec91e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:36.650676 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab8e9d-7c70-40e9-ab0d-aa40404ec91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:36.668431 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c5ab8e9d-7c70-40e9-ab0d-aa40404ec91e None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:36.669631 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c5ab8e9d-7c70-40e9-ab0d-aa40404ec91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:36.728135 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1432/2866] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:36.737780 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1634966-fc60-41fe-9548-4d212bebf765 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:36.739166 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-e1634966-fc60-41fe-9548-4d212bebf765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:36.795786 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1634966-fc60-41fe-9548-4d212bebf765 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:36.799521 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27222b10-dde1-4b42-9b0d-11a9569a10d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:36.800199 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222b10-dde1-4b42-9b0d-11a9569a10d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:36.800541 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222b10-dde1-4b42-9b0d-11a9569a10d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:36.800863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222b10-dde1-4b42-9b0d-11a9569a10d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:36.811845 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27222b10-dde1-4b42-9b0d-11a9569a10d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:36.813292 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27222b10-dde1-4b42-9b0d-11a9569a10d3 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:36.822766 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1435/2867] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:37.095075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab8e9d-7c70-40e9-ab0d-aa40404ec91e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:37.096532 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c5ab8e9d-7c70-40e9-ab0d-aa40404ec91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:37.112114 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab8e9d-7c70-40e9-ab0d-aa40404ec91e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:37.117502 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841aefd6-ef7f-423f-8c88-cc6b37056f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:37.118366 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841aefd6-ef7f-423f-8c88-cc6b37056f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:37.118754 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841aefd6-ef7f-423f-8c88-cc6b37056f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:37.118926 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841aefd6-ef7f-423f-8c88-cc6b37056f57 admin admin] PATH_INFO: `/v3/projects/9458b930c75e457398a48b73f88847d6` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:37.120157 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841aefd6-ef7f-423f-8c88-cc6b37056f57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9458b930c75e457398a48b73f88847d6)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:37.122104 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841aefd6-ef7f-423f-8c88-cc6b37056f57 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:37.171060 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-841aefd6-ef7f-423f-8c88-cc6b37056f57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9458b930c75e457398a48b73f88847d6', 'request_id': 'req-841aefd6-ef7f-423f-8c88-cc6b37056f57'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:37.177158 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-841aefd6-ef7f-423f-8c88-cc6b37056f57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9458b930c75e457398a48b73f88847d6', 'request_id': 'req-841aefd6-ef7f-423f-8c88-cc6b37056f57'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:37.186009 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-841aefd6-ef7f-423f-8c88-cc6b37056f57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9458b930c75e457398a48b73f88847d6', 'request_id': 'req-841aefd6-ef7f-423f-8c88-cc6b37056f57'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:37.197340 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-841aefd6-ef7f-423f-8c88-cc6b37056f57 admin admin] The token cache is being invalidate because project 9458b930c75e457398a48b73f88847d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:37.197632 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-841aefd6-ef7f-423f-8c88-cc6b37056f57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:37.198287 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-841aefd6-ef7f-423f-8c88-cc6b37056f57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:37.199824 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1433/2868] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:37 2022] DELETE /identity/v3/projects/9458b930c75e457398a48b73f88847d6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:37.204798 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242306fd-62ff-4ea2-a956-2d5c1e56072b None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:37.205084 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242306fd-62ff-4ea2-a956-2d5c1e56072b None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:37.205359 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242306fd-62ff-4ea2-a956-2d5c1e56072b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:37.223139 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-242306fd-62ff-4ea2-a956-2d5c1e56072b None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:37.224384 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-242306fd-62ff-4ea2-a956-2d5c1e56072b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:37.241292 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:37.284355 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1436/2869] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:37.292367 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007c04e-fe30-4b55-87d8-4342d4d20217 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:37.292660 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007c04e-fe30-4b55-87d8-4342d4d20217 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:37.292939 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007c04e-fe30-4b55-87d8-4342d4d20217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:37.313054 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-9007c04e-fe30-4b55-87d8-4342d4d20217 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:37.314281 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9007c04e-fe30-4b55-87d8-4342d4d20217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:37.323699 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1434/2870] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:37.328038 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242306fd-62ff-4ea2-a956-2d5c1e56072b None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:37.338243 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242306fd-62ff-4ea2-a956-2d5c1e56072b None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:37.343333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ea7b1-4be7-4ea1-a441-e468f22e98f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:37.344010 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ea7b1-4be7-4ea1-a441-e468f22e98f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:37.344311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ea7b1-4be7-4ea1-a441-e468f22e98f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:37.344612 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ea7b1-4be7-4ea1-a441-e468f22e98f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:37.345362 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ea7b1-4be7-4ea1-a441-e468f22e98f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80242) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:36:37.345671 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ea7b1-4be7-4ea1-a441-e468f22e98f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:37.347301 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ea7b1-4be7-4ea1-a441-e468f22e98f8 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:37.360505 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1437/2871] 10.222.0.43 () {58 vars in 1152 bytes} [Wed Jul 20 19:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:36:37.364988 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007c04e-fe30-4b55-87d8-4342d4d20217 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:37.374936 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007c04e-fe30-4b55-87d8-4342d4d20217 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:37.379563 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef42fde-be19-4b3d-af54-aad3e630c657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:37.380471 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef42fde-be19-4b3d-af54-aad3e630c657 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:37.380694 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef42fde-be19-4b3d-af54-aad3e630c657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:37.381124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef42fde-be19-4b3d-af54-aad3e630c657 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:37.382508 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef42fde-be19-4b3d-af54-aad3e630c657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:37.384533 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef42fde-be19-4b3d-af54-aad3e630c657 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:37.410771 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1435/2872] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:37 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:37.415067 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ea7b1-4be7-4ea1-a441-e468f22e98f8 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:37.426171 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ea7b1-4be7-4ea1-a441-e468f22e98f8 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:37.487180 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33552d27-72cb-46ae-a80f-b0ba1309f0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:37.487842 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33552d27-72cb-46ae-a80f-b0ba1309f0d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:37.488128 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33552d27-72cb-46ae-a80f-b0ba1309f0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:37.488451 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33552d27-72cb-46ae-a80f-b0ba1309f0d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:37.489131 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33552d27-72cb-46ae-a80f-b0ba1309f0d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:37.491599 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33552d27-72cb-46ae-a80f-b0ba1309f0d0 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:37.511337 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1438/2873] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:37 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:36:37.515345 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef42fde-be19-4b3d-af54-aad3e630c657 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:37.525549 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef42fde-be19-4b3d-af54-aad3e630c657 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:37.530112 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22813fe-818d-430d-b3e4-30f9947ba5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:37.530767 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22813fe-818d-430d-b3e4-30f9947ba5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:37.531087 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22813fe-818d-430d-b3e4-30f9947ba5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:37.531361 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22813fe-818d-430d-b3e4-30f9947ba5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:37.532080 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22813fe-818d-430d-b3e4-30f9947ba5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:37.533714 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22813fe-818d-430d-b3e4-30f9947ba5c4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:37.544588 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1436/2874] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:37.548533 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33552d27-72cb-46ae-a80f-b0ba1309f0d0 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:37.559481 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33552d27-72cb-46ae-a80f-b0ba1309f0d0 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:37.564346 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84633d22-e1d3-448a-b7ff-28576a984892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:37.564996 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84633d22-e1d3-448a-b7ff-28576a984892 admin admin] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:37.565275 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84633d22-e1d3-448a-b7ff-28576a984892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:37.565546 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84633d22-e1d3-448a-b7ff-28576a984892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:37.566287 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84633d22-e1d3-448a-b7ff-28576a984892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:37.567988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84633d22-e1d3-448a-b7ff-28576a984892 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:37.578278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1439/2875] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:36:37.582429 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22813fe-818d-430d-b3e4-30f9947ba5c4 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:37.592613 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22813fe-818d-430d-b3e4-30f9947ba5c4 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:37.597246 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db702fde-e7db-43b6-a998-5a6818ca94b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:37.598114 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db702fde-e7db-43b6-a998-5a6818ca94b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:37.598396 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db702fde-e7db-43b6-a998-5a6818ca94b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:37.598677 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db702fde-e7db-43b6-a998-5a6818ca94b3 admin admin] PATH_INFO: `/v3/projects/887ce1159e3c412b9cab436ecf76737c/users/1d423c82039a41cd96a2c8b25c37647f/roles/18f0dc41b78c4b52b6912225928ea388` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:37.613628 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db702fde-e7db-43b6-a998-5a6818ca94b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=887ce1159e3c412b9cab436ecf76737c, user_id=1d423c82039a41cd96a2c8b25c37647f, role_id=18f0dc41b78c4b52b6912225928ea388)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:37.616964 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db702fde-e7db-43b6-a998-5a6818ca94b3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:37.629788 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1437/2876] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:37 2022] PUT /identity/v3/projects/887ce1159e3c412b9cab436ecf76737c/users/1d423c82039a41cd96a2c8b25c37647f/roles/18f0dc41b78c4b52b6912225928ea388 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:37.635353 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191450e9-328f-48c7-8f3e-6b4809b1f628 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:37.635671 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191450e9-328f-48c7-8f3e-6b4809b1f628 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:37.635934 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191450e9-328f-48c7-8f3e-6b4809b1f628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:37.653436 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-191450e9-328f-48c7-8f3e-6b4809b1f628 None None] MFA Rules not processed for user `44b032994ab44ba0acf66376eaab2451`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:37.654632 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-191450e9-328f-48c7-8f3e-6b4809b1f628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:37.713873 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1440/2877] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:37.723525 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db702fde-e7db-43b6-a998-5a6818ca94b3 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:37.725904 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-db702fde-e7db-43b6-a998-5a6818ca94b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:37.747981 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:37.784826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db702fde-e7db-43b6-a998-5a6818ca94b3 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:37.789036 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba9d62-528a-4969-9d6c-1571d51ab434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:37.789684 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba9d62-528a-4969-9d6c-1571d51ab434 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:37.789972 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba9d62-528a-4969-9d6c-1571d51ab434 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:37.790248 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba9d62-528a-4969-9d6c-1571d51ab434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:37.799236 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba9d62-528a-4969-9d6c-1571d51ab434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:37.800735 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba9d62-528a-4969-9d6c-1571d51ab434 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:37.811058 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1438/2878] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:41.924279 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b483bf-1e25-4219-9131-8da3f6dde813 None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:41.924817 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b483bf-1e25-4219-9131-8da3f6dde813 None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:41.924817 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b483bf-1e25-4219-9131-8da3f6dde813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:41.954970 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-68b483bf-1e25-4219-9131-8da3f6dde813 None None] MFA Rules not processed for user `1d423c82039a41cd96a2c8b25c37647f`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:41.956511 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-68b483bf-1e25-4219-9131-8da3f6dde813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:41.975894 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:42.028135 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1441/2879] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:36:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:42.032748 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba9d62-528a-4969-9d6c-1571d51ab434 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:42.043248 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba9d62-528a-4969-9d6c-1571d51ab434 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:42.048088 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a51da7-27eb-47fa-8f0e-9e5d39ba2e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:42.048845 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a51da7-27eb-47fa-8f0e-9e5d39ba2e4a admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:42.049133 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a51da7-27eb-47fa-8f0e-9e5d39ba2e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:42.049432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a51da7-27eb-47fa-8f0e-9e5d39ba2e4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:42.050130 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a51da7-27eb-47fa-8f0e-9e5d39ba2e4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:42.052054 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a51da7-27eb-47fa-8f0e-9e5d39ba2e4a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:42.079550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1439/2880] 10.222.0.43 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:42 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:42.083576 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b483bf-1e25-4219-9131-8da3f6dde813 None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:42.095048 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b483bf-1e25-4219-9131-8da3f6dde813 None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:42.152835 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cafc55-45d7-4183-ad69-37de556e277b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:42.153523 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cafc55-45d7-4183-ad69-37de556e277b admin admin] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:42.153808 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cafc55-45d7-4183-ad69-37de556e277b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:42.154083 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cafc55-45d7-4183-ad69-37de556e277b admin admin] PATH_INFO: `/v3/users` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:42.154786 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cafc55-45d7-4183-ad69-37de556e277b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:42.156725 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cafc55-45d7-4183-ad69-37de556e277b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:42.179026 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1442/2881] 10.222.0.43 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:42 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:42.183070 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a51da7-27eb-47fa-8f0e-9e5d39ba2e4a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:42.194365 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a51da7-27eb-47fa-8f0e-9e5d39ba2e4a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:42.199146 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6af62-f46d-4a3f-9459-868df219e1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:42.199836 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6af62-f46d-4a3f-9459-868df219e1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:42.200124 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6af62-f46d-4a3f-9459-868df219e1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:42.200449 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6af62-f46d-4a3f-9459-868df219e1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:42.201770 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6af62-f46d-4a3f-9459-868df219e1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:42.203554 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6af62-f46d-4a3f-9459-868df219e1c1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:42.215056 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1440/2882] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:42.220403 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cafc55-45d7-4183-ad69-37de556e277b admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:42.231470 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cafc55-45d7-4183-ad69-37de556e277b admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:42.236172 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb1d3b-fa5e-4c04-ae0b-000e8a011c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:42.237091 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb1d3b-fa5e-4c04-ae0b-000e8a011c23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:42.237374 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb1d3b-fa5e-4c04-ae0b-000e8a011c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:42.237651 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb1d3b-fa5e-4c04-ae0b-000e8a011c23 admin admin] PATH_INFO: `/v3/projects/7349ccbe72944fbca1592e1e7fe8fb42/users/365ecb612da8490ca61465f883e67404/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:42.254719 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb1d3b-fa5e-4c04-ae0b-000e8a011c23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7349ccbe72944fbca1592e1e7fe8fb42, user_id=365ecb612da8490ca61465f883e67404, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:42.257462 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb1d3b-fa5e-4c04-ae0b-000e8a011c23 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:42.274341 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1443/2883] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:42 2022] PUT /identity/v3/projects/7349ccbe72944fbca1592e1e7fe8fb42/users/365ecb612da8490ca61465f883e67404/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:42.278437 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6af62-f46d-4a3f-9459-868df219e1c1 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:42.289227 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6af62-f46d-4a3f-9459-868df219e1c1 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:42.350805 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8f1a3-d6dd-4e10-8233-dc8571b6d2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:42.351634 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8f1a3-d6dd-4e10-8233-dc8571b6d2de admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:42.352015 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8f1a3-d6dd-4e10-8233-dc8571b6d2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:42.352426 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8f1a3-d6dd-4e10-8233-dc8571b6d2de admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:42.353289 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8f1a3-d6dd-4e10-8233-dc8571b6d2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:42.355050 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8f1a3-d6dd-4e10-8233-dc8571b6d2de admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:42.366439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1441/2884] 10.222.0.43 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:42.370518 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb1d3b-fa5e-4c04-ae0b-000e8a011c23 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:42.381520 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb1d3b-fa5e-4c04-ae0b-000e8a011c23 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:42.386109 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b1679-5429-4fa6-86ea-8ca77492928b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:42.386990 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b1679-5429-4fa6-86ea-8ca77492928b admin admin] REQUEST_METHOD: `PUT` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:42.387278 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b1679-5429-4fa6-86ea-8ca77492928b admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:42.387558 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b1679-5429-4fa6-86ea-8ca77492928b admin admin] PATH_INFO: `/v3/projects/7349ccbe72944fbca1592e1e7fe8fb42/users/365ecb612da8490ca61465f883e67404/roles/8c52108bb1d34ca2a1d0fc643095dae8` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:42.389569 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b1679-5429-4fa6-86ea-8ca77492928b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7349ccbe72944fbca1592e1e7fe8fb42, user_id=365ecb612da8490ca61465f883e67404, role_id=8c52108bb1d34ca2a1d0fc643095dae8)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:42.392188 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b1679-5429-4fa6-86ea-8ca77492928b admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:42.402741 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1444/2885] 10.222.0.43 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:42 2022] PUT /identity/v3/projects/7349ccbe72944fbca1592e1e7fe8fb42/users/365ecb612da8490ca61465f883e67404/roles/8c52108bb1d34ca2a1d0fc643095dae8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:47.368942 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c79e47-2ee5-4629-b1d1-dbe9fa139b2d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:47.369491 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c79e47-2ee5-4629-b1d1-dbe9fa139b2d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:47.369597 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c79e47-2ee5-4629-b1d1-dbe9fa139b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:47.405682 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-33c79e47-2ee5-4629-b1d1-dbe9fa139b2d None None] MFA Rules not processed for user `365ecb612da8490ca61465f883e67404`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:47.406938 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-33c79e47-2ee5-4629-b1d1-dbe9fa139b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:47.511702 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1442/2886] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:36:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:47.530562 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b1679-5429-4fa6-86ea-8ca77492928b admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:47.532988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-eb3b1679-5429-4fa6-86ea-8ca77492928b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:47.619096 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b1679-5429-4fa6-86ea-8ca77492928b admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:47.622896 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae8d61-7109-4da5-b293-7145a6db0937 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3e28557896e41c893f0de2590bc96d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:47.623981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae8d61-7109-4da5-b293-7145a6db0937 service glance] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:47.623981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae8d61-7109-4da5-b293-7145a6db0937 service glance] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:47.624264 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae8d61-7109-4da5-b293-7145a6db0937 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:47.660186 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:47.701069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae8d61-7109-4da5-b293-7145a6db0937 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:47.702504 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae8d61-7109-4da5-b293-7145a6db0937 service glance] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:47.714214 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1445/2887] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:47.829932 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db522ce-c127-4775-8758-683df23324b0 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:47.830225 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db522ce-c127-4775-8758-683df23324b0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:47.830526 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db522ce-c127-4775-8758-683df23324b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:47.848151 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:47.898905 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4db522ce-c127-4775-8758-683df23324b0 None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:47.900033 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4db522ce-c127-4775-8758-683df23324b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:47.969797 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1443/2888] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jul 20 19:36:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:48.018458 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae8d61-7109-4da5-b293-7145a6db0937 service glance] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:48.020070 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-a2ae8d61-7109-4da5-b293-7145a6db0937 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:48.115840 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae8d61-7109-4da5-b293-7145a6db0937 service glance] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:48.123735 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2253807-a9b4-47bf-a389-b759c03475bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:48.124898 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2253807-a9b4-47bf-a389-b759c03475bb service swift] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:48.125445 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2253807-a9b4-47bf-a389-b759c03475bb service swift] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:48.126116 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2253807-a9b4-47bf-a389-b759c03475bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:48.136853 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2253807-a9b4-47bf-a389-b759c03475bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:48.138443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2253807-a9b4-47bf-a389-b759c03475bb service swift] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:48.146310 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1446/2889] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:36:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:36:56.799843 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955ad25-17ee-4952-b753-7888e5616b6d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:56.800439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955ad25-17ee-4952-b753-7888e5616b6d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:56.800439 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955ad25-17ee-4952-b753-7888e5616b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:56.818075 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1955ad25-17ee-4952-b753-7888e5616b6d None None] MFA Rules not processed for user `1d423c82039a41cd96a2c8b25c37647f`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:56.819280 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1955ad25-17ee-4952-b753-7888e5616b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:56.826936 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1444/2890] 10.222.0.43 () {58 vars in 957 bytes} [Wed Jul 20 19:36:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:57.077423 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2253807-a9b4-47bf-a389-b759c03475bb service swift] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:57.131370 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2253807-a9b4-47bf-a389-b759c03475bb service swift] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:57.134972 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564fd93-0b46-4835-9087-3b7a2961ef63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:57.135621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564fd93-0b46-4835-9087-3b7a2961ef63 service neutron] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:57.135907 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564fd93-0b46-4835-9087-3b7a2961ef63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:57.136187 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564fd93-0b46-4835-9087-3b7a2961ef63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:57.138021 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-0564fd93-0b46-4835-9087-3b7a2961ef63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:57.148348 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564fd93-0b46-4835-9087-3b7a2961ef63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:57.149797 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564fd93-0b46-4835-9087-3b7a2961ef63 service neutron] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:57.158005 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1447/2891] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:36:57.498598 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955ad25-17ee-4952-b753-7888e5616b6d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:57.500109 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1955ad25-17ee-4952-b753-7888e5616b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:57.574029 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955ad25-17ee-4952-b753-7888e5616b6d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:57.578978 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5071be-06a7-4336-95ae-33a2e4721017 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae581d3dda354e07894cb5205d1b2c5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:57.579624 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5071be-06a7-4336-95ae-33a2e4721017 service placement] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:57.579906 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5071be-06a7-4336-95ae-33a2e4721017 service placement] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:57.580189 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5071be-06a7-4336-95ae-33a2e4721017 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:57.581983 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ce5071be-06a7-4336-95ae-33a2e4721017 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:57.593854 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5071be-06a7-4336-95ae-33a2e4721017 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:57.595424 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5071be-06a7-4336-95ae-33a2e4721017 service placement] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:57.605368 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1445/2892] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:36:58.837060 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9a033-b710-440b-9bdf-a57e56a63fff None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:58.837584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9a033-b710-440b-9bdf-a57e56a63fff None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:58.837584 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9a033-b710-440b-9bdf-a57e56a63fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:58.855829 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-9ce9a033-b710-440b-9bdf-a57e56a63fff None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:58.857189 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-9ce9a033-b710-440b-9bdf-a57e56a63fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:58.865976 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1448/2893] 10.222.0.43 () {58 vars in 1009 bytes} [Wed Jul 20 19:36:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:58.882884 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5071be-06a7-4336-95ae-33a2e4721017 service placement] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:58.894180 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5071be-06a7-4336-95ae-33a2e4721017 service placement] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:58.900848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc0af5-f40f-4df8-97b9-3137cc3cecc7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:58.901556 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc0af5-f40f-4df8-97b9-3137cc3cecc7 service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:58.901846 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc0af5-f40f-4df8-97b9-3137cc3cecc7 service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:58.902122 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc0af5-f40f-4df8-97b9-3137cc3cecc7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:58.911592 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc0af5-f40f-4df8-97b9-3137cc3cecc7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:58.913814 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc0af5-f40f-4df8-97b9-3137cc3cecc7 service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:58.923506 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1446/2894] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:36:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:37:08.850951 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=86 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:08.919391 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=80 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:12.890652 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9a033-b710-440b-9bdf-a57e56a63fff None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:12.892596 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-9ce9a033-b710-440b-9bdf-a57e56a63fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:12.980124 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9a033-b710-440b-9bdf-a57e56a63fff None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:12.986751 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8caf96-f656-42e6-ba09-752a3196717d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:12.987403 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8caf96-f656-42e6-ba09-752a3196717d service nova] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:12.987686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8caf96-f656-42e6-ba09-752a3196717d service nova] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:12.987961 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8caf96-f656-42e6-ba09-752a3196717d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:12.990686 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-cb8caf96-f656-42e6-ba09-752a3196717d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:13.003333 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8caf96-f656-42e6-ba09-752a3196717d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:13.004980 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8caf96-f656-42e6-ba09-752a3196717d service nova] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:13.015602 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1449/2895] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:37:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:37:23.011069 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=34 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:24.282174 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc0af5-f40f-4df8-97b9-3137cc3cecc7 service swift] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:24.283605 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-45fc0af5-f40f-4df8-97b9-3137cc3cecc7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:24.296048 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc0af5-f40f-4df8-97b9-3137cc3cecc7 service swift] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:24.299550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09541595-e233-4a50-ab3f-c575743215c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74327a10eaab43e0af02050da1a24f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:24.300179 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09541595-e233-4a50-ab3f-c575743215c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:24.300495 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09541595-e233-4a50-ab3f-c575743215c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:24.300761 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09541595-e233-4a50-ab3f-c575743215c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:24.310408 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09541595-e233-4a50-ab3f-c575743215c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:24.311821 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09541595-e233-4a50-ab3f-c575743215c4 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:24.321432 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1447/2896] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:37:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:37:34.317173 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=6 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:55.456221 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8caf96-f656-42e6-ba09-752a3196717d service nova] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:55.467311 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8caf96-f656-42e6-ba09-752a3196717d service nova] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:55.472225 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5117ea-5bfc-4b24-ad5b-66577d0f0b69 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae581d3dda354e07894cb5205d1b2c5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:55.472877 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5117ea-5bfc-4b24-ad5b-66577d0f0b69 service placement] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:55.473158 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5117ea-5bfc-4b24-ad5b-66577d0f0b69 service placement] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:55.473432 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5117ea-5bfc-4b24-ad5b-66577d0f0b69 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:55.475664 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-7e5117ea-5bfc-4b24-ad5b-66577d0f0b69 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:55.486882 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5117ea-5bfc-4b24-ad5b-66577d0f0b69 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:55.488443 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5117ea-5bfc-4b24-ad5b-66577d0f0b69 service placement] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:55.498863 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1450/2897] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:37:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:38:05.494724 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=6 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:38:35.221083 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09541595-e233-4a50-ab3f-c575743215c4 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:35.222654 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-09541595-e233-4a50-ab3f-c575743215c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:35.253012 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09541595-e233-4a50-ab3f-c575743215c4 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:35.260848 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270be7c-a616-42c7-9888-7ef8158f07a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267b178236e046e4bc694b2da6f86eb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:35.261520 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270be7c-a616-42c7-9888-7ef8158f07a3 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:35.261804 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270be7c-a616-42c7-9888-7ef8158f07a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:35.262081 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270be7c-a616-42c7-9888-7ef8158f07a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:35.264534 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f270be7c-a616-42c7-9888-7ef8158f07a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:35.279962 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270be7c-a616-42c7-9888-7ef8158f07a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:35.281607 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270be7c-a616-42c7-9888-7ef8158f07a3 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:35.294692 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1448/2898] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:38:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:38:42.472649 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da991035-d428-4336-8f57-f775a5a2cfcd None None] REQUEST_METHOD: `POST` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:42.473243 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da991035-d428-4336-8f57-f775a5a2cfcd None None] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:42.473243 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da991035-d428-4336-8f57-f775a5a2cfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:42.491375 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.auth.core [None req-da991035-d428-4336-8f57-f775a5a2cfcd None None] MFA Rules not processed for user `2d514236db644010859fc18250f5894e`. Rule list: `[]` (Enabled: `True`). {{(pid=80242) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:38:42.492621 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-da991035-d428-4336-8f57-f775a5a2cfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:42.499788 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1451/2899] 10.222.0.43 () {58 vars in 967 bytes} [Wed Jul 20 19:38:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:38:42.540064 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270be7c-a616-42c7-9888-7ef8158f07a3 service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:42.553374 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270be7c-a616-42c7-9888-7ef8158f07a3 service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:42.559670 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc1667-2a43-4bda-a8c2-9ab8b7e49c7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c944eab8a8948749bb8375787216b09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:42.560342 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc1667-2a43-4bda-a8c2-9ab8b7e49c7f service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:42.560716 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc1667-2a43-4bda-a8c2-9ab8b7e49c7f service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:42.560998 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc1667-2a43-4bda-a8c2-9ab8b7e49c7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:42.571108 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc1667-2a43-4bda-a8c2-9ab8b7e49c7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:42.572714 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc1667-2a43-4bda-a8c2-9ab8b7e49c7f service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:42.580826 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1449/2900] 10.222.0.43 () {60 vars in 1384 bytes} [Wed Jul 20 19:38:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:38:45.971395 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da991035-d428-4336-8f57-f775a5a2cfcd None None] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:45.986293 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da991035-d428-4336-8f57-f775a5a2cfcd None None] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:45.995768 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3eb07-75bf-43de-83b3-68d066e6b916 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae581d3dda354e07894cb5205d1b2c5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:45.996914 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3eb07-75bf-43de-83b3-68d066e6b916 service placement] REQUEST_METHOD: `GET` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:45.997218 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3eb07-75bf-43de-83b3-68d066e6b916 service placement] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:45.997500 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3eb07-75bf-43de-83b3-68d066e6b916 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:45.999471 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-4cf3eb07-75bf-43de-83b3-68d066e6b916 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:46.014068 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3eb07-75bf-43de-83b3-68d066e6b916 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:46.015834 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3eb07-75bf-43de-83b3-68d066e6b916 service placement] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:46.044417 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1452/2901] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:38:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:38:49.496395 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc1667-2a43-4bda-a8c2-9ab8b7e49c7f service swift] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:49.692310 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc1667-2a43-4bda-a8c2-9ab8b7e49c7f service swift] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:49.696941 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c637d-065a-4e3d-ab01-fab074908e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:49.697866 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c637d-065a-4e3d-ab01-fab074908e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:49.698156 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c637d-065a-4e3d-ab01-fab074908e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:49.698436 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c637d-065a-4e3d-ab01-fab074908e66 admin admin] PATH_INFO: `/v3/users/1d423c82039a41cd96a2c8b25c37647f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:49.699648 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c637d-065a-4e3d-ab01-fab074908e66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d423c82039a41cd96a2c8b25c37647f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:49.701685 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c637d-065a-4e3d-ab01-fab074908e66 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:49.750602 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b19c637d-065a-4e3d-ab01-fab074908e66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d423c82039a41cd96a2c8b25c37647f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:49.759008 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b19c637d-065a-4e3d-ab01-fab074908e66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d423c82039a41cd96a2c8b25c37647f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:49.759631 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b19c637d-065a-4e3d-ab01-fab074908e66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d423c82039a41cd96a2c8b25c37647f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:49.766768 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b19c637d-065a-4e3d-ab01-fab074908e66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d423c82039a41cd96a2c8b25c37647f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:49.780131 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b19c637d-065a-4e3d-ab01-fab074908e66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d423c82039a41cd96a2c8b25c37647f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:49.786857 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1450/2902] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:38:49 2022] DELETE /identity/v3/users/1d423c82039a41cd96a2c8b25c37647f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:38:49.982254 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3eb07-75bf-43de-83b3-68d066e6b916 service placement] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:49.993889 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3eb07-75bf-43de-83b3-68d066e6b916 service placement] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:50.052849 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:50.053724 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:50.054009 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:50.054353 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3 admin admin] PATH_INFO: `/v3/projects/887ce1159e3c412b9cab436ecf76737c` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:50.055547 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=887ce1159e3c412b9cab436ecf76737c)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:50.057756 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3 admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:50.111170 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '887ce1159e3c412b9cab436ecf76737c', 'request_id': 'req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:50.119349 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '887ce1159e3c412b9cab436ecf76737c', 'request_id': 'req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:50.127363 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '887ce1159e3c412b9cab436ecf76737c', 'request_id': 'req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:50.139661 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3 admin admin] The token cache is being invalidate because project 887ce1159e3c412b9cab436ecf76737c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:38:50.139952 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:50.140744 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:50.142201 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1453/2903] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:38:49 2022] DELETE /identity/v3/projects/887ce1159e3c412b9cab436ecf76737c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:38:50.147009 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c637d-065a-4e3d-ab01-fab074908e66 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:50.148490 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b19c637d-065a-4e3d-ab01-fab074908e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:50.162185 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c637d-065a-4e3d-ab01-fab074908e66 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:50.209987 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:38:50.220020 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52211bef-63a5-4687-94d6-00515d74384e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:50.220927 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52211bef-63a5-4687-94d6-00515d74384e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:50.221401 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52211bef-63a5-4687-94d6-00515d74384e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:50.221678 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52211bef-63a5-4687-94d6-00515d74384e admin admin] PATH_INFO: `/v3/users/365ecb612da8490ca61465f883e67404` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:50.222866 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52211bef-63a5-4687-94d6-00515d74384e admin admin] RBAC: Authorizing `identity:delete_user(user_id=365ecb612da8490ca61465f883e67404)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:50.224833 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52211bef-63a5-4687-94d6-00515d74384e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:50.277334 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-52211bef-63a5-4687-94d6-00515d74384e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '365ecb612da8490ca61465f883e67404'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:50.285086 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-52211bef-63a5-4687-94d6-00515d74384e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '365ecb612da8490ca61465f883e67404'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:50.285776 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-52211bef-63a5-4687-94d6-00515d74384e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '365ecb612da8490ca61465f883e67404'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:50.292764 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-52211bef-63a5-4687-94d6-00515d74384e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '365ecb612da8490ca61465f883e67404'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:50.304531 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-52211bef-63a5-4687-94d6-00515d74384e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '365ecb612da8490ca61465f883e67404'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:50.310786 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1451/2904] 10.222.0.43 () {58 vars in 1223 bytes} [Wed Jul 20 19:38:50 2022] DELETE /identity/v3/users/365ecb612da8490ca61465f883e67404 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:38:50.486567 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3 admin admin] Authenticating user token {{(pid=80242) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:50.488867 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.fernet_utils [None req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80242) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:50.503433 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dcd6b-1a8e-4922-88c6-e68c82aaf6e3 admin admin] Validating token access rules against request {{(pid=80242) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:50.563296 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be13eb1-a5a4-48b1-a0e5-ec6cf41391ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44b032994ab44ba0acf66376eaab2451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39f878600fda47389da52621d6e0947c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80242) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:50.564334 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be13eb1-a5a4-48b1-a0e5-ec6cf41391ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:50.564676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be13eb1-a5a4-48b1-a0e5-ec6cf41391ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:50.564966 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be13eb1-a5a4-48b1-a0e5-ec6cf41391ce admin admin] PATH_INFO: `/v3/projects/7349ccbe72944fbca1592e1e7fe8fb42` {{(pid=80242) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:50.566148 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be13eb1-a5a4-48b1-a0e5-ec6cf41391ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=7349ccbe72944fbca1592e1e7fe8fb42)` {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:50.568676 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be13eb1-a5a4-48b1-a0e5-ec6cf41391ce admin admin] RBAC: Authorization granted {{(pid=80242) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:50.580361 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:38:50.618988 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-3be13eb1-a5a4-48b1-a0e5-ec6cf41391ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7349ccbe72944fbca1592e1e7fe8fb42', 'request_id': 'req-3be13eb1-a5a4-48b1-a0e5-ec6cf41391ce'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:50.626557 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-3be13eb1-a5a4-48b1-a0e5-ec6cf41391ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7349ccbe72944fbca1592e1e7fe8fb42', 'request_id': 'req-3be13eb1-a5a4-48b1-a0e5-ec6cf41391ce'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:50.632981 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-3be13eb1-a5a4-48b1-a0e5-ec6cf41391ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7349ccbe72944fbca1592e1e7fe8fb42', 'request_id': 'req-3be13eb1-a5a4-48b1-a0e5-ec6cf41391ce'} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:50.644818 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-3be13eb1-a5a4-48b1-a0e5-ec6cf41391ce admin admin] The token cache is being invalidate because project 7349ccbe72944fbca1592e1e7fe8fb42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80242) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:38:50.645110 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-3be13eb1-a5a4-48b1-a0e5-ec6cf41391ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:50.645983 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG keystone.notifications [None req-3be13eb1-a5a4-48b1-a0e5-ec6cf41391ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80242) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:50.647557 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: [pid: 80242|app: 0|req: 1454/2905] 10.222.0.43 () {58 vars in 1232 bytes} [Wed Jul 20 19:38:50 2022] DELETE /identity/v3/projects/7349ccbe72944fbca1592e1e7fe8fb42 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:39:00.307313 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:39:00.641620 dv-cinder-819149-14-iscsi devstack@keystone.service[80242]: DEBUG dbcounter [-] [80242] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:43:00.430089 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52211bef-63a5-4687-94d6-00515d74384e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:43:00.431496 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-52211bef-63a5-4687-94d6-00515d74384e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:43:00.570550 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52211bef-63a5-4687-94d6-00515d74384e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:43:00.630565 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fe21a-8892-4ab7-913d-db60a5bb48ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae581d3dda354e07894cb5205d1b2c5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e2a50ebbb48417c911df499d98a0157', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:43:00.631216 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fe21a-8892-4ab7-913d-db60a5bb48ca service placement] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:43:00.631502 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fe21a-8892-4ab7-913d-db60a5bb48ca service placement] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:43:00.631779 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fe21a-8892-4ab7-913d-db60a5bb48ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:43:00.633828 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ff6fe21a-8892-4ab7-913d-db60a5bb48ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:43:00.645988 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6fe21a-8892-4ab7-913d-db60a5bb48ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:43:00.647792 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6fe21a-8892-4ab7-913d-db60a5bb48ca service placement] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:43:00.658127 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1452/2906] 10.222.0.43 () {60 vars in 1365 bytes} [Wed Jul 20 19:43:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:43:10.653564 dv-cinder-819149-14-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}